Jul 06 11:28:50.363422 ip-10-0-143-173 systemd[1]: Starting Kubernetes Kubelet... Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.697820 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702758 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702767 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702771 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702774 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702779 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702783 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 11:28:50.761417 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702787 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702791 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702794 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702796 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702799 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702807 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702810 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702813 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702816 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702818 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702821 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702823 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702834 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702837 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702840 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702843 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702845 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702848 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702850 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 11:28:50.762400 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702855 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702858 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702861 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702864 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702867 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702869 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702873 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702876 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702879 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702882 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702884 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702887 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702889 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702892 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702894 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702896 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702899 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702901 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702904 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702906 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 11:28:50.763031 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702909 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702911 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702915 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702918 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702921 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702924 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702927 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702929 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702932 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702934 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702937 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702939 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702942 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702946 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702949 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702951 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702954 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702956 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702958 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702961 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 11:28:50.763728 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702964 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702966 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702969 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702971 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702973 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702976 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702978 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702981 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702983 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702985 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702988 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702990 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702992 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702995 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702997 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.702999 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703002 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703005 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703007 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703009 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 11:28:50.765378 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703012 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703014 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703017 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703019 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703022 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703024 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703027 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703029 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703031 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703034 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703036 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703038 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703041 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703044 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703733 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703740 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703743 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703746 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703749 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 11:28:50.767146 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703753 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703755 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703758 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703761 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703764 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703766 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703769 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703772 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703776 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703778 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703781 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703784 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703787 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703789 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703792 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703794 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703797 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703800 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703802 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703805 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 11:28:50.768069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703807 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703809 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703812 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703814 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703817 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703819 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703822 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703826 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703828 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703831 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703833 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703835 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703838 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703840 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703843 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703845 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703848 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703850 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703853 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 11:28:50.770185 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703855 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703858 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703860 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703863 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703865 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703868 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703871 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703873 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703875 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703877 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703880 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703883 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703885 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703887 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703890 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703892 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703894 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703897 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703899 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703901 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703905 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 11:28:50.772501 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703907 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703912 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703915 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703918 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703921 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703925 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703927 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703930 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703932 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703934 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703937 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703939 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703942 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703944 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703946 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703950 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703953 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703957 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703959 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 11:28:50.776134 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703963 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703966 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703969 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703972 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703974 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703976 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703979 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703981 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703984 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703986 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703988 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703991 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703993 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703996 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.703999 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704350 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704354 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704357 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704360 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704363 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 11:28:50.780674 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704365 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704368 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704370 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704373 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704375 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704378 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704380 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704383 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704385 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704388 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704390 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704393 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704395 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704398 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704401 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704403 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704406 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704408 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704411 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704414 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 11:28:50.781712 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704416 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704419 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704421 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704424 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704426 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704429 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704432 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704434 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704437 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704439 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704442 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704444 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704446 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704449 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704451 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704453 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704456 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704458 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704460 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704463 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704465 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 11:28:50.782531 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704467 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704470 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704474 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704479 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704482 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704485 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704487 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704490 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704493 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704497 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704500 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704503 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704505 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704508 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704510 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704513 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704515 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704520 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704523 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 11:28:50.783137 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704525 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704528 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704530 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704532 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704535 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704537 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704540 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704542 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704545 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704549 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704551 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704554 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704556 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704559 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704561 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704563 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704566 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704568 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704573 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 11:28:50.783745 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704576 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704579 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704582 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704584 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704587 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704590 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704592 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704595 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704598 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704600 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704602 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704606 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704609 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704611 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.704614 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704672 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704678 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704683 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704687 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704690 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704693 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 11:28:50.784350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704696 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704701 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704704 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704707 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704710 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704713 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704716 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704719 2585 flags.go:64] FLAG: --cgroup-root="" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704722 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704725 2585 flags.go:64] FLAG: --client-ca-file="" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704727 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704730 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704733 2585 flags.go:64] FLAG: --cluster-domain="" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704736 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704739 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704741 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704745 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704753 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704755 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704758 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704761 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704764 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704769 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704771 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 11:28:50.784956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704780 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704783 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704791 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704794 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704797 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704800 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704802 2585 flags.go:64] FLAG: --enable-server="true" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704805 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704809 2585 flags.go:64] FLAG: --event-burst="100" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704813 2585 flags.go:64] FLAG: --event-qps="50" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704816 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704819 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704822 2585 flags.go:64] FLAG: --eviction-hard="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704825 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704828 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704830 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704833 2585 flags.go:64] FLAG: --eviction-soft="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704836 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704838 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704841 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704844 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704846 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704849 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704852 2585 flags.go:64] FLAG: --feature-gates="" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704855 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 11:28:50.785597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704858 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704860 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704863 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704866 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704869 2585 flags.go:64] FLAG: --help="false" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704873 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-143-173.ec2.internal" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704876 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.704878 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705242 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705246 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705249 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705252 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705255 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705258 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705261 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705263 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705269 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705272 2585 flags.go:64] FLAG: --kube-reserved="" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705275 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705277 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705280 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705283 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705286 2585 flags.go:64] FLAG: --lock-file="" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705288 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 11:28:50.786310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705291 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705294 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705299 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705301 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705304 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705306 2585 flags.go:64] FLAG: --logging-format="text" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705309 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705312 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705315 2585 flags.go:64] FLAG: --manifest-url="" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705317 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705321 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705324 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705327 2585 flags.go:64] FLAG: --max-pods="110" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705331 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705334 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705336 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705339 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705342 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705345 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705348 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705355 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705357 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705361 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 11:28:50.787293 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705365 2585 flags.go:64] FLAG: --pod-cidr="" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705371 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705378 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705382 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705386 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705390 2585 flags.go:64] FLAG: --port="10250" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705395 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705398 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07d19670e3f442348" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705401 2585 flags.go:64] FLAG: --qos-reserved="" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705404 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705407 2585 flags.go:64] FLAG: --register-node="true" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705410 2585 flags.go:64] FLAG: --register-with-taints="" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705413 2585 flags.go:64] FLAG: --registry-burst="10" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705416 2585 flags.go:64] FLAG: --registry-qps="5" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705418 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705421 2585 flags.go:64] FLAG: --reserved-memory="" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705425 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705427 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705430 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705433 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705436 2585 flags.go:64] FLAG: --runonce="false" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705439 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705441 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705446 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705448 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 11:28:50.787955 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705451 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705454 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705457 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705460 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705463 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705466 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705468 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705471 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705475 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705477 2585 flags.go:64] FLAG: --system-cgroups="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705480 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705485 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705488 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705490 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705493 2585 flags.go:64] FLAG: --tls-min-version="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705496 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705499 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705501 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705504 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705507 2585 flags.go:64] FLAG: --v="2" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705510 2585 flags.go:64] FLAG: --version="false" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705514 2585 flags.go:64] FLAG: --vmodule="" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705517 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.705520 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705621 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 11:28:50.872659 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705624 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 11:28:50.789451 ip-10-0-143-173 systemd[1]: Started Kubernetes Kubelet. Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705627 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705630 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705633 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705635 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705639 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705641 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705646 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705649 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705652 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705655 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705658 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705660 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705663 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705665 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705669 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705672 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705674 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705676 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705679 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 11:28:50.873483 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705681 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705684 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705687 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705689 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705691 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705694 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705696 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705699 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705701 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705703 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705706 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705708 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705711 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705713 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705715 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705718 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705720 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705724 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705727 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705729 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 11:28:50.874187 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705732 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705734 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705736 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705739 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705742 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705744 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705746 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705750 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705754 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705757 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705761 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705763 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705766 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705769 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705771 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705774 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705776 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705778 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705781 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 11:28:50.874851 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705783 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705785 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705788 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705790 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705792 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705795 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705797 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705800 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705802 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705805 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705808 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705811 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705813 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705816 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705818 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705821 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705823 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705826 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705828 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705830 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 11:28:50.875477 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705834 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705837 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705840 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705844 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705846 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705849 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705851 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705853 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705856 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705858 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705861 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705863 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705865 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705868 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705870 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705873 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705875 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705877 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 11:28:50.876101 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.705880 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.706397 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.713533 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.713545 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.714142 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.714151 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714217 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714221 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714225 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714228 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714232 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714236 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714239 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714242 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 11:28:50.876649 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714247 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714250 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714252 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714255 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714258 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714261 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714263 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714266 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714268 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714270 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714273 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714275 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714278 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714280 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714283 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714285 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714288 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714290 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714293 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714296 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 11:28:50.877119 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714298 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714301 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714303 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714307 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714309 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714312 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714315 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714317 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714320 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714322 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714325 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714327 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714331 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714334 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714336 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714340 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714343 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714346 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714349 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 11:28:50.877755 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714351 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714354 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714356 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714359 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714361 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714363 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714366 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714369 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714371 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714374 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714376 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714379 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714381 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714384 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714386 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714389 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714393 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714396 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714398 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714400 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 11:28:50.878372 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714403 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714405 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714408 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714410 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714413 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714416 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714419 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714422 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714425 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714427 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714430 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714432 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714436 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714440 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714443 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714446 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714449 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714451 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714454 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 11:28:50.878939 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714456 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714459 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714461 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714464 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714467 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714469 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714471 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714474 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714477 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714480 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714483 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714485 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714488 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.714492 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714621 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 11:28:50.879527 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714626 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714629 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714632 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714637 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714640 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714643 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714646 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714649 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714652 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714656 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714660 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714663 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714666 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714669 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714672 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714674 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714677 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714680 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714683 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714685 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 11:28:50.879967 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714688 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714691 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714693 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714696 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714698 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714701 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714703 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714707 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714709 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714712 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714714 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714717 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714719 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714722 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714724 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714728 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714730 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714733 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714735 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714738 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 11:28:50.880547 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714740 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714743 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714745 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714748 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714751 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714753 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714755 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714758 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714760 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714764 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714767 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714769 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714772 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714774 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714777 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714779 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714781 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714784 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714786 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 11:28:50.881064 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714791 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714794 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714796 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714798 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714801 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714803 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714805 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714808 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714811 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714814 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714816 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714819 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714821 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714823 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714833 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714836 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714838 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714841 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714848 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714850 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 11:28:50.881646 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714853 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714855 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714858 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714861 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714865 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714868 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714870 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714873 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714875 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714878 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714880 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714883 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714886 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714889 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714891 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714894 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714896 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714898 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 11:28:50.882174 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:50.714901 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.714905 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.715005 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.717234 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.718579 2585 server.go:1019] "Starting client certificate rotation" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.718671 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.718703 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.720728 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.721731 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.734495 2585 log.go:25] "Validated CRI v1 runtime API" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.736539 2585 log.go:25] "Validated CRI v1 image API" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.738246 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 11:28:50.882675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.740238 2585 fs.go:135] Filesystem UUIDs: map[626f3528-6379-438f-be84-6575b35d8728:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d164137e-f26c-4534-9d61-f66ae6d990a6:/dev/nvme0n1p4] Jul 06 11:28:50.885042 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.740251 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 11:28:50.885042 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.745314 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.746590 2585 manager.go:216] Machine: {Timestamp:2026-07-06 11:28:50.745001066 +0000 UTC m=+0.276740384 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099057 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d81a334af09cf79c595c37dabf812 SystemUUID:ec2d81a3-34af-09cf-79c5-95c37dabf812 BootID:0f9999c2-f12b-4601-88c2-cfd854ef834e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:4c:19:62:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:4c:19:62:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:24:f8:b6:c7:25 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.746703 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.746813 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.749595 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.749621 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.750220 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.750231 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.750238 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.750853 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.752140 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756075 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756186 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756210 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756218 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756231 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756239 2585 kubelet.go:398] "Adding apiserver pod source" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756246 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 11:28:50.888936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756512 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.756531 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.758242 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.760391 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.760445 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764475 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764491 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764498 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764505 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764516 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764522 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764532 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764538 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764546 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.764552 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.785176 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.785200 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.785230 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.785231 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.785888 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.785902 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.785911 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7jdns" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.785926 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7jdns" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.787834 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.788837 2585 server.go:1268] "Started kubelet" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.788903 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.788963 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.789196 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.790671 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.790946 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.794189 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.795739 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.795814 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.795906 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796334 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796375 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796383 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796447 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796457 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796675 2585 factory.go:55] Registering systemd factory Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796689 2585 factory.go:223] Registration of the systemd container factory successfully Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796894 2585 factory.go:153] Registering CRI-O factory Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796906 2585 factory.go:223] Registration of the crio container factory successfully Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796956 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796982 2585 factory.go:103] Registering Raw factory Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.796997 2585 manager.go:1196] Started watching for new ooms in manager Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.797626 2585 manager.go:349] Starting recovery of all containers Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.804526 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:50.889515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.808789 2585 manager.go:354] Recovery completed Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.810281 2585 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-173.ec2.internal\" not found" node="ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.812630 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814410 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814433 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814443 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814837 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814850 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814859 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.814868 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.815424 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.815442 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.815452 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.817026 2585 policy_none.go:49] "None policy: Start" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.817040 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.817054 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.819088 2585 policy_none.go:47] "Start" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.857710 2585 manager.go:337] "Starting Device Plugin manager" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.857838 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.857850 2585 server.go:87] "Starting device plugin registration server" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.858039 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.858050 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.858138 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.858228 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.858238 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.858842 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.858876 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.958334 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.960450 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.960476 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.960493 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.960521 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.970354 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.970369 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-173.ec2.internal\": node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.989822 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.991615 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.992700 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.992737 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.992756 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:50.992792 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 11:28:51.039734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:50.995522 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.090722 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.093859 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal"] Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.093907 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.095314 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.095335 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.095348 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.097518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa33dbc8162804a9c2f54b7dac53c409-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal\" (UID: \"fa33dbc8162804a9c2f54b7dac53c409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.097538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa33dbc8162804a9c2f54b7dac53c409-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal\" (UID: \"fa33dbc8162804a9c2f54b7dac53c409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.097681 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.097791 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.097814 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.098277 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.098281 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.098321 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.098331 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.098304 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.098382 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.100610 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.100628 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.106902 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientMemory" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.106920 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.106929 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeHasSufficientPID" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.126643 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-173.ec2.internal\" not found" node="ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.132342 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-173.ec2.internal\" not found" node="ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.191387 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.198504 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/088e17fe59b4ec12c1114c83bdf80edd-config\") pod \"kube-apiserver-proxy-ip-10-0-143-173.ec2.internal\" (UID: \"088e17fe59b4ec12c1114c83bdf80edd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.198537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa33dbc8162804a9c2f54b7dac53c409-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal\" (UID: \"fa33dbc8162804a9c2f54b7dac53c409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.219366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.198554 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa33dbc8162804a9c2f54b7dac53c409-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal\" (UID: \"fa33dbc8162804a9c2f54b7dac53c409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.222139 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.198601 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa33dbc8162804a9c2f54b7dac53c409-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal\" (UID: \"fa33dbc8162804a9c2f54b7dac53c409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.222139 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.198611 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa33dbc8162804a9c2f54b7dac53c409-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal\" (UID: \"fa33dbc8162804a9c2f54b7dac53c409\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.418507 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.293380 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.418507 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.299239 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/088e17fe59b4ec12c1114c83bdf80edd-config\") pod \"kube-apiserver-proxy-ip-10-0-143-173.ec2.internal\" (UID: \"088e17fe59b4ec12c1114c83bdf80edd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.418507 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.299270 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/088e17fe59b4ec12c1114c83bdf80edd-config\") pod \"kube-apiserver-proxy-ip-10-0-143-173.ec2.internal\" (UID: \"088e17fe59b4ec12c1114c83bdf80edd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.418507 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.401203 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.559799 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.430535 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.559799 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.440153 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.559799 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.501303 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.603315 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.672327 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.704353 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.717949 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.718078 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.718093 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 11:28:51.720338 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.718082 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 11:28:51.867867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.790664 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 11:23:51 +0000 UTC" deadline="2027-12-04 05:39:02.571200517 +0000 UTC" Jul 06 11:28:51.867867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.790690 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12378h10m10.780514512s" Jul 06 11:28:51.867867 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:51.804518 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-173.ec2.internal\" not found" Jul 06 11:28:51.980313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.878788 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:51.980313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.897394 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.980313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.910425 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 11:28:51.980313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.911427 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" Jul 06 11:28:51.980313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:51.921495 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 11:28:52.242367 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:52.233867 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088e17fe59b4ec12c1114c83bdf80edd.slice/crio-b0b43db6cc6f92f7426530ecfa9fa69276c3e875065340fcafcdd6480c54f5f8 WatchSource:0}: Error finding container b0b43db6cc6f92f7426530ecfa9fa69276c3e875065340fcafcdd6480c54f5f8: Status 404 returned error can't find the container with id b0b43db6cc6f92f7426530ecfa9fa69276c3e875065340fcafcdd6480c54f5f8 Jul 06 11:28:52.242367 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.239692 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 11:28:52.247940 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:52.247111 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa33dbc8162804a9c2f54b7dac53c409.slice/crio-ab470cdb7af09607b1c36f3937dc2141416dccc421578332e6857ee1e9790d43 WatchSource:0}: Error finding container ab470cdb7af09607b1c36f3937dc2141416dccc421578332e6857ee1e9790d43: Status 404 returned error can't find the container with id ab470cdb7af09607b1c36f3937dc2141416dccc421578332e6857ee1e9790d43 Jul 06 11:28:52.759492 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.759207 2585 apiserver.go:52] "Watching apiserver" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.765111 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.765578 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-qhh2l","kube-system/global-pull-secret-syncer-q8q7d","kube-system/konnectivity-agent-fsvg9","openshift-cluster-node-tuning-operator/tuned-nghnk","openshift-dns/node-resolver-8lck9","openshift-image-registry/node-ca-5qngh","openshift-multus/multus-additional-cni-plugins-8n4tf","openshift-network-diagnostics/network-check-target-66m6t","openshift-ovn-kubernetes/ovnkube-node-bcrwd","kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal","openshift-multus/multus-vwx7g","openshift-multus/network-metrics-daemon-pfkrn"] Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.769650 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.772148 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.773354 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.773560 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.773804 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.774045 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.774074 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h6jzn\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.774353 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.774408 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.774698 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.775062 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tc98b\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.777209 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.777538 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-srk5b\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.777713 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.777891 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.779921 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.781439 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7kl2r\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.781667 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.781713 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.782158 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.782335 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.782403 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.782750 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zs6xs\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.784745 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.784911 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.784921 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.785506 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b4smc\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.785513 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.788601 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.788757 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.788837 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.788916 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 11:28:52.821260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.788970 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.789035 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7gg98\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.788761 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.790804 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.793041 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.793064 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.793292 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pbxtx\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.793479 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.795114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.795190 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.796410 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.798127 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.798228 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.800167 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.802296 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.802517 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j8v4v\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.802774 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.802852 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.802905 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.802958 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.809339 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813299 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-daemon-config\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813332 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-etc-kubernetes\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813361 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-kubernetes\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813386 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-cni-netd\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.823562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813413 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-ovnkube-config\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813453 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-hostroot\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813497 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813532 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-systemd\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813566 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-netns\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813604 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-cni-bin\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813648 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6v9m\" (UniqueName: \"kubernetes.io/projected/fc73ff93-6a36-47fb-b2f0-e85088d70097-kube-api-access-q6v9m\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813677 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a6ad56c-e987-47b6-a64c-acf6705a3228-kubelet-config\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813707 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysctl-d\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-etc-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813757 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-multus-certs\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813799 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5b8fa1d-0de0-4339-bfc7-1e940522963c-agent-certs\") pod \"konnectivity-agent-fsvg9\" (UID: \"b5b8fa1d-0de0-4339-bfc7-1e940522963c\") " pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813824 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50752636-200c-4d95-ad03-1c19fdc20645-hosts-file\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813864 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-run\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813892 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-var-lib-kubelet\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813937 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-run-ovn-kubernetes\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.813969 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1068e2de-db09-452c-9b12-8ba49d8cbb33-host-slash\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.831006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814007 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-os-release\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814032 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cnibin\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814057 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysctl-conf\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814214 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-cni-bin\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814251 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w9vw\" (UniqueName: \"kubernetes.io/projected/1068e2de-db09-452c-9b12-8ba49d8cbb33-kube-api-access-2w9vw\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814316 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcmrd\" (UniqueName: \"kubernetes.io/projected/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-kube-api-access-rcmrd\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814351 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-tuned\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814379 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnl4m\" (UniqueName: \"kubernetes.io/projected/50752636-200c-4d95-ad03-1c19fdc20645-kube-api-access-lnl4m\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814431 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-system-cni-dir\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814456 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814485 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-sys\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814507 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30711795-85a6-4b52-aa6b-06a1188f7b58-tmp\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814530 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-conf-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814554 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86rqx\" (UniqueName: \"kubernetes.io/projected/30711795-85a6-4b52-aa6b-06a1188f7b58-kube-api-access-86rqx\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.832262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814578 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-slash\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814602 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/48fea88c-3651-4ef7-9403-3dc063c0468d-ovn-node-metrics-cert\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814628 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814650 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-host\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-systemd-units\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814699 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87p6n\" (UniqueName: \"kubernetes.io/projected/48fea88c-3651-4ef7-9403-3dc063c0468d-kube-api-access-87p6n\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814723 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-k8s-cni-cncf-io\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814748 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-cni-multus\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814773 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50752636-200c-4d95-ad03-1c19fdc20645-tmp-dir\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814830 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-host\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814870 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-ovnkube-script-lib\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814896 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-socket-dir-parent\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5b8fa1d-0de0-4339-bfc7-1e940522963c-konnectivity-ca\") pod \"konnectivity-agent-fsvg9\" (UID: \"b5b8fa1d-0de0-4339-bfc7-1e940522963c\") " pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814969 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-os-release\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.814994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-modprobe-d\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815017 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-systemd\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.833251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815060 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-kubelet\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815172 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-var-lib-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815242 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-log-socket\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815331 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-env-overrides\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-kubelet\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815403 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxv2\" (UniqueName: \"kubernetes.io/projected/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-kube-api-access-jfxv2\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-ovn\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815461 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-system-cni-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815507 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-cni-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815540 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysconfig\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815568 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-lib-modules\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815597 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815622 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1068e2de-db09-452c-9b12-8ba49d8cbb33-iptables-alerter-script\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815652 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-cnibin\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc73ff93-6a36-47fb-b2f0-e85088d70097-cni-binary-copy\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815735 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-serviceca\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815772 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a6ad56c-e987-47b6-a64c-acf6705a3228-dbus\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.834907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815814 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.836102 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815839 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.836102 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.815935 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-run-netns\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.836102 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.816758 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-node-log\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.836102 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.816795 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc64q\" (UniqueName: \"kubernetes.io/projected/672720fa-209d-4354-8698-27024f8e88ec-kube-api-access-kc64q\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:52.844593 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.844524 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ckpf" Jul 06 11:28:52.851720 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.851173 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ckpf" Jul 06 11:28:52.888004 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.887967 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:52.897499 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.897469 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 11:28:52.917643 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.917613 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5b8fa1d-0de0-4339-bfc7-1e940522963c-agent-certs\") pod \"konnectivity-agent-fsvg9\" (UID: \"b5b8fa1d-0de0-4339-bfc7-1e940522963c\") " pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.917758 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.917656 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50752636-200c-4d95-ad03-1c19fdc20645-hosts-file\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.917822 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.917726 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50752636-200c-4d95-ad03-1c19fdc20645-hosts-file\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.917887 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.917830 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-registration-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.917887 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.917858 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-run\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918003 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.917973 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-var-lib-kubelet\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918060 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918039 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-run\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918060 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918009 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-run-ovn-kubernetes\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918173 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918043 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-var-lib-kubelet\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918173 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918051 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 11:28:52.918173 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918110 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1068e2de-db09-452c-9b12-8ba49d8cbb33-host-slash\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.918173 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918080 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1068e2de-db09-452c-9b12-8ba49d8cbb33-host-slash\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-os-release\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918231 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cnibin\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-run-ovn-kubernetes\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918261 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918289 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysctl-conf\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918308 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-os-release\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918287 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cnibin\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918314 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-cni-bin\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w9vw\" (UniqueName: \"kubernetes.io/projected/1068e2de-db09-452c-9b12-8ba49d8cbb33-kube-api-access-2w9vw\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918380 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcmrd\" (UniqueName: \"kubernetes.io/projected/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-kube-api-access-rcmrd\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.918403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918370 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-cni-bin\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918410 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-device-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-tuned\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918494 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnl4m\" (UniqueName: \"kubernetes.io/projected/50752636-200c-4d95-ad03-1c19fdc20645-kube-api-access-lnl4m\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918520 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-system-cni-dir\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918576 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-sys\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918605 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30711795-85a6-4b52-aa6b-06a1188f7b58-tmp\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918614 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918631 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-conf-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918412 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysctl-conf\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86rqx\" (UniqueName: \"kubernetes.io/projected/30711795-85a6-4b52-aa6b-06a1188f7b58-kube-api-access-86rqx\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918680 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-slash\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918705 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/48fea88c-3651-4ef7-9403-3dc063c0468d-ovn-node-metrics-cert\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918733 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918756 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-host\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.918936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-systemd-units\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918805 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87p6n\" (UniqueName: \"kubernetes.io/projected/48fea88c-3651-4ef7-9403-3dc063c0468d-kube-api-access-87p6n\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918832 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-k8s-cni-cncf-io\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918860 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-cni-multus\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918861 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918884 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50752636-200c-4d95-ad03-1c19fdc20645-tmp-dir\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918895 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-system-cni-dir\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918922 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-host\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918951 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-ovnkube-script-lib\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.918976 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-socket-dir-parent\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919002 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5b8fa1d-0de0-4339-bfc7-1e940522963c-konnectivity-ca\") pod \"konnectivity-agent-fsvg9\" (UID: \"b5b8fa1d-0de0-4339-bfc7-1e940522963c\") " pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919029 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-sys-fs\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-os-release\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919081 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-modprobe-d\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919104 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-systemd\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919133 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-kubelet\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-var-lib-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.919854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919186 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-log-socket\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919212 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50752636-200c-4d95-ad03-1c19fdc20645-tmp-dir\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919211 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-env-overrides\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-kubelet\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919336 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxv2\" (UniqueName: \"kubernetes.io/projected/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-kube-api-access-jfxv2\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-ovn\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919404 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-system-cni-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919457 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.919431 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-cni-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.920246 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-host\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.920417 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-sys\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.920687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.920558 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-env-overrides\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.921527 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-ovn\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.921980 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-host\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922056 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-socket-dir-parent\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922112 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-var-lib-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922227 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-kubelet\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922347 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-log-socket\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922390 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-systemd-units\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922434 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-conf-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922449 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-ovnkube-script-lib\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922477 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysconfig\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922509 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-lib-modules\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922595 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922632 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1068e2de-db09-452c-9b12-8ba49d8cbb33-iptables-alerter-script\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.923020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.922664 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-cnibin\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.923071 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc73ff93-6a36-47fb-b2f0-e85088d70097-cni-binary-copy\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.923136 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-slash\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.923637 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1068e2de-db09-452c-9b12-8ba49d8cbb33-iptables-alerter-script\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.923628 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-systemd\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.923714 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-kubelet\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.923739 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:52.923863 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.923847 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:28:53.423808335 +0000 UTC m=+2.955547663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:52.924223 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924004 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysconfig\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924395 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-cni-multus\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924533 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-cni-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924584 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-serviceca\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a6ad56c-e987-47b6-a64c-acf6705a3228-dbus\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924844 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc73ff93-6a36-47fb-b2f0-e85088d70097-cni-binary-copy\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924864 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-cnibin\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924954 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-run-netns\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.924990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-node-log\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925027 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc64q\" (UniqueName: \"kubernetes.io/projected/672720fa-209d-4354-8698-27024f8e88ec-kube-api-access-kc64q\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925066 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-daemon-config\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925098 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-etc-kubernetes\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925130 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-kubernetes\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925134 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-k8s-cni-cncf-io\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925163 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-cni-netd\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-ovnkube-config\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.926638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925265 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-modprobe-d\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925069 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30711795-85a6-4b52-aa6b-06a1188f7b58-tmp\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925391 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925398 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-node-log\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925444 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-etc-kubernetes\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925478 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-hostroot\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925525 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925559 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925598 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckf2h\" (UniqueName: \"kubernetes.io/projected/4b09df2b-9cbd-447a-b3ee-6d3101207218-kube-api-access-ckf2h\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925695 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-socket-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925733 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925730 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-kubernetes\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925797 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-tuned\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-systemd\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925916 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-netns\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925941 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-os-release\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.927613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.925974 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-cni-bin\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926010 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6v9m\" (UniqueName: \"kubernetes.io/projected/fc73ff93-6a36-47fb-b2f0-e85088d70097-kube-api-access-q6v9m\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926046 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a6ad56c-e987-47b6-a64c-acf6705a3228-kubelet-config\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926053 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-lib-modules\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926077 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysctl-d\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-etc-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.926174 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926183 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-multus-certs\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926202 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc73ff93-6a36-47fb-b2f0-e85088d70097-multus-daemon-config\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926220 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-cni-netd\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926515 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5b8fa1d-0de0-4339-bfc7-1e940522963c-konnectivity-ca\") pod \"konnectivity-agent-fsvg9\" (UID: \"b5b8fa1d-0de0-4339-bfc7-1e940522963c\") " pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:52.926556 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:28:53.426513244 +0000 UTC m=+2.958252551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926566 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a6ad56c-e987-47b6-a64c-acf6705a3228-kubelet-config\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926584 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-multus-certs\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926547 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-etc-openvswitch\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926651 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-hostroot\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926720 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30711795-85a6-4b52-aa6b-06a1188f7b58-etc-sysctl-d\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.928445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.926953 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-system-cni-dir\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.927316 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/48fea88c-3651-4ef7-9403-3dc063c0468d-ovnkube-config\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.927413 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-run-netns\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.927878 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.927888 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.927947 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-run-systemd\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.928006 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48fea88c-3651-4ef7-9403-3dc063c0468d-host-run-netns\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.928191 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc73ff93-6a36-47fb-b2f0-e85088d70097-host-var-lib-cni-bin\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.928387 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a6ad56c-e987-47b6-a64c-acf6705a3228-dbus\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.928495 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-serviceca\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.929218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.928950 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5b8fa1d-0de0-4339-bfc7-1e940522963c-agent-certs\") pod \"konnectivity-agent-fsvg9\" (UID: \"b5b8fa1d-0de0-4339-bfc7-1e940522963c\") " pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:52.929735 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.929701 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/48fea88c-3651-4ef7-9403-3dc063c0468d-ovn-node-metrics-cert\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.941455 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.941430 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w9vw\" (UniqueName: \"kubernetes.io/projected/1068e2de-db09-452c-9b12-8ba49d8cbb33-kube-api-access-2w9vw\") pod \"iptables-alerter-qhh2l\" (UID: \"1068e2de-db09-452c-9b12-8ba49d8cbb33\") " pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:52.941718 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.941679 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnl4m\" (UniqueName: \"kubernetes.io/projected/50752636-200c-4d95-ad03-1c19fdc20645-kube-api-access-lnl4m\") pod \"node-resolver-8lck9\" (UID: \"50752636-200c-4d95-ad03-1c19fdc20645\") " pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:52.942094 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.942072 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcmrd\" (UniqueName: \"kubernetes.io/projected/7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3-kube-api-access-rcmrd\") pod \"node-ca-5qngh\" (UID: \"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3\") " pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:52.960215 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.960173 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6v9m\" (UniqueName: \"kubernetes.io/projected/fc73ff93-6a36-47fb-b2f0-e85088d70097-kube-api-access-q6v9m\") pod \"multus-vwx7g\" (UID: \"fc73ff93-6a36-47fb-b2f0-e85088d70097\") " pod="openshift-multus/multus-vwx7g" Jul 06 11:28:52.961122 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.961094 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc64q\" (UniqueName: \"kubernetes.io/projected/672720fa-209d-4354-8698-27024f8e88ec-kube-api-access-kc64q\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:52.961318 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.961296 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86rqx\" (UniqueName: \"kubernetes.io/projected/30711795-85a6-4b52-aa6b-06a1188f7b58-kube-api-access-86rqx\") pod \"tuned-nghnk\" (UID: \"30711795-85a6-4b52-aa6b-06a1188f7b58\") " pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:52.961965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.961919 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87p6n\" (UniqueName: \"kubernetes.io/projected/48fea88c-3651-4ef7-9403-3dc063c0468d-kube-api-access-87p6n\") pod \"ovnkube-node-bcrwd\" (UID: \"48fea88c-3651-4ef7-9403-3dc063c0468d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:52.963277 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.963170 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxv2\" (UniqueName: \"kubernetes.io/projected/49de6972-f0b8-4bc0-a83d-b837a63e3a6f-kube-api-access-jfxv2\") pod \"multus-additional-cni-plugins-8n4tf\" (UID: \"49de6972-f0b8-4bc0-a83d-b837a63e3a6f\") " pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:52.998970 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:52.998925 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" event={"ID":"fa33dbc8162804a9c2f54b7dac53c409","Type":"ContainerStarted","Data":"ab470cdb7af09607b1c36f3937dc2141416dccc421578332e6857ee1e9790d43"} Jul 06 11:28:53.013079 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.013049 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" event={"ID":"088e17fe59b4ec12c1114c83bdf80edd","Type":"ContainerStarted","Data":"b0b43db6cc6f92f7426530ecfa9fa69276c3e875065340fcafcdd6480c54f5f8"} Jul 06 11:28:53.027126 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027084 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027126 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027119 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027305 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027152 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckf2h\" (UniqueName: \"kubernetes.io/projected/4b09df2b-9cbd-447a-b3ee-6d3101207218-kube-api-access-ckf2h\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027305 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027188 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-socket-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027305 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:53.027305 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027243 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027305 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-registration-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027564 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027310 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-device-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027564 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027368 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-sys-fs\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027564 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027484 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-registration-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027564 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027516 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-socket-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027564 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027548 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-device-dir\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027891 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027594 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.027891 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.027650 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b09df2b-9cbd-447a-b3ee-6d3101207218-sys-fs\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.034687 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.034355 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:28:53.034687 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.034389 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:28:53.034687 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.034403 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:53.034687 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.034525 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:28:53.534478973 +0000 UTC m=+3.066218290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:53.037918 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.037890 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckf2h\" (UniqueName: \"kubernetes.io/projected/4b09df2b-9cbd-447a-b3ee-6d3101207218-kube-api-access-ckf2h\") pod \"aws-ebs-csi-driver-node-p2p5n\" (UID: \"4b09df2b-9cbd-447a-b3ee-6d3101207218\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.088203 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.088078 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" Jul 06 11:28:53.093403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.093379 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vwx7g" Jul 06 11:28:53.100744 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.100718 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49de6972_f0b8_4bc0_a83d_b837a63e3a6f.slice/crio-6223a3255928b58c3599880bdd108d13b26593332a788365860d6139f2796498 WatchSource:0}: Error finding container 6223a3255928b58c3599880bdd108d13b26593332a788365860d6139f2796498: Status 404 returned error can't find the container with id 6223a3255928b58c3599880bdd108d13b26593332a788365860d6139f2796498 Jul 06 11:28:53.103400 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.103381 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:28:53.104978 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.104950 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc73ff93_6a36_47fb_b2f0_e85088d70097.slice/crio-0dc2e106855291fb9b21df80e8483dc093f942a659bb1fa3bae1da028437d69f WatchSource:0}: Error finding container 0dc2e106855291fb9b21df80e8483dc093f942a659bb1fa3bae1da028437d69f: Status 404 returned error can't find the container with id 0dc2e106855291fb9b21df80e8483dc093f942a659bb1fa3bae1da028437d69f Jul 06 11:28:53.110439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.110303 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nghnk" Jul 06 11:28:53.115336 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.115311 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b8fa1d_0de0_4339_bfc7_1e940522963c.slice/crio-487acbe338ec844429ff212a8a3efea41f49b274bfbf76b73b13ad582b59cb32 WatchSource:0}: Error finding container 487acbe338ec844429ff212a8a3efea41f49b274bfbf76b73b13ad582b59cb32: Status 404 returned error can't find the container with id 487acbe338ec844429ff212a8a3efea41f49b274bfbf76b73b13ad582b59cb32 Jul 06 11:28:53.119395 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.119232 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8lck9" Jul 06 11:28:53.121791 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.121769 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30711795_85a6_4b52_aa6b_06a1188f7b58.slice/crio-57b80ba5a52fba7a4c736d09e594240acfd64fe6122e4e71784ee81c43f95b5e WatchSource:0}: Error finding container 57b80ba5a52fba7a4c736d09e594240acfd64fe6122e4e71784ee81c43f95b5e: Status 404 returned error can't find the container with id 57b80ba5a52fba7a4c736d09e594240acfd64fe6122e4e71784ee81c43f95b5e Jul 06 11:28:53.130926 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.130900 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5qngh" Jul 06 11:28:53.131836 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.131813 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50752636_200c_4d95_ad03_1c19fdc20645.slice/crio-d691beaa99f6125fcb4c4e311e35b0e72df1a4c51b1d55ddd246dabf8c02fdd2 WatchSource:0}: Error finding container d691beaa99f6125fcb4c4e311e35b0e72df1a4c51b1d55ddd246dabf8c02fdd2: Status 404 returned error can't find the container with id d691beaa99f6125fcb4c4e311e35b0e72df1a4c51b1d55ddd246dabf8c02fdd2 Jul 06 11:28:53.141163 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.141143 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:28:53.142180 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.142141 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bda73a4_7ae5_4c45_b4af_e4a49bb5d9a3.slice/crio-d7762bf7023e11a32670e6c62eba2fc3acb0ac5bdeac234753706a44678c85c9 WatchSource:0}: Error finding container d7762bf7023e11a32670e6c62eba2fc3acb0ac5bdeac234753706a44678c85c9: Status 404 returned error can't find the container with id d7762bf7023e11a32670e6c62eba2fc3acb0ac5bdeac234753706a44678c85c9 Jul 06 11:28:53.151789 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:53.151767 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48fea88c_3651_4ef7_9403_3dc063c0468d.slice/crio-881a2baacd02df87cae517968e74b9ca40b57cb4f7442dc142fd70abe1f1ef11 WatchSource:0}: Error finding container 881a2baacd02df87cae517968e74b9ca40b57cb4f7442dc142fd70abe1f1ef11: Status 404 returned error can't find the container with id 881a2baacd02df87cae517968e74b9ca40b57cb4f7442dc142fd70abe1f1ef11 Jul 06 11:28:53.189701 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.189684 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qhh2l" Jul 06 11:28:53.197675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.197651 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" Jul 06 11:28:53.317055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.316972 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:53.334152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.334119 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 11:28:53.430728 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.430642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:53.430859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.430734 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:53.430859 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.430821 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:53.430859 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.430846 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:53.430994 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.430898 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:28:54.430881999 +0000 UTC m=+3.962621304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:53.430994 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.430921 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:28:54.430909138 +0000 UTC m=+3.962648449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:53.632007 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.631968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:53.632321 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.632160 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:28:53.632321 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.632195 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:28:53.632321 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.632205 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:53.632321 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.632259 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:28:54.632244576 +0000 UTC m=+4.163983882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:53.855167 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.855075 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 11:23:53 +0000 UTC" deadline="2028-04-12 04:25:33.928543116 +0000 UTC" Jul 06 11:28:53.855167 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.855117 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15496h56m40.073430689s" Jul 06 11:28:53.993740 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:53.993211 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:53.993740 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:53.993351 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:28:54.016587 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.016541 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"881a2baacd02df87cae517968e74b9ca40b57cb4f7442dc142fd70abe1f1ef11"} Jul 06 11:28:54.017952 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.017919 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5qngh" event={"ID":"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3","Type":"ContainerStarted","Data":"d7762bf7023e11a32670e6c62eba2fc3acb0ac5bdeac234753706a44678c85c9"} Jul 06 11:28:54.019585 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.019557 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8lck9" event={"ID":"50752636-200c-4d95-ad03-1c19fdc20645","Type":"ContainerStarted","Data":"d691beaa99f6125fcb4c4e311e35b0e72df1a4c51b1d55ddd246dabf8c02fdd2"} Jul 06 11:28:54.021779 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.021753 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fsvg9" event={"ID":"b5b8fa1d-0de0-4339-bfc7-1e940522963c","Type":"ContainerStarted","Data":"487acbe338ec844429ff212a8a3efea41f49b274bfbf76b73b13ad582b59cb32"} Jul 06 11:28:54.025352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.025303 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nghnk" event={"ID":"30711795-85a6-4b52-aa6b-06a1188f7b58","Type":"ContainerStarted","Data":"57b80ba5a52fba7a4c736d09e594240acfd64fe6122e4e71784ee81c43f95b5e"} Jul 06 11:28:54.027883 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.027860 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vwx7g" event={"ID":"fc73ff93-6a36-47fb-b2f0-e85088d70097","Type":"ContainerStarted","Data":"0dc2e106855291fb9b21df80e8483dc093f942a659bb1fa3bae1da028437d69f"} Jul 06 11:28:54.029866 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.029820 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerStarted","Data":"6223a3255928b58c3599880bdd108d13b26593332a788365860d6139f2796498"} Jul 06 11:28:54.440122 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.440083 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:54.440332 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.440153 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:54.440332 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.440322 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:54.440461 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.440339 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:54.440461 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.440405 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:28:56.440383415 +0000 UTC m=+5.972122739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:54.440461 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.440430 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:28:56.440417262 +0000 UTC m=+5.972156567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:54.642168 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.642116 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:54.642447 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.642427 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:28:54.642555 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.642475 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:28:54.642555 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.642494 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:54.642555 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.642555 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:28:56.642535349 +0000 UTC m=+6.174274671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:54.855946 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.855875 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 11:23:53 +0000 UTC" deadline="2028-04-01 06:01:42.793434046 +0000 UTC" Jul 06 11:28:54.855946 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.855943 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15234h32m47.937495373s" Jul 06 11:28:54.994428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.994351 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:54.994895 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.994479 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:28:54.996498 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:54.996332 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:54.996498 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:54.996440 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:28:55.993760 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:55.993254 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:55.993760 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:55.993402 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:28:56.456688 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:56.456655 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:56.457149 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:56.456725 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:56.457149 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.456867 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:56.457149 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.456928 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:00.456909857 +0000 UTC m=+9.988649178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:28:56.457451 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.457360 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:56.457451 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.457418 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:29:00.457400676 +0000 UTC m=+9.989139994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:28:56.658676 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:56.658634 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:56.658841 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.658828 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:28:56.658902 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.658852 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:28:56.658902 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.658865 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:56.658996 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.658921 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:00.658903507 +0000 UTC m=+10.190642831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:28:56.993606 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:56.993578 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:56.993797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:56.993579 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:56.993797 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.993710 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:28:56.993906 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:56.993808 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:28:57.943978 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:57.943938 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b09df2b_9cbd_447a_b3ee_6d3101207218.slice/crio-b939620e8d61442a7745d9733db1c8e8fc90f9b1b798612ce2293b7e39fbf87d WatchSource:0}: Error finding container b939620e8d61442a7745d9733db1c8e8fc90f9b1b798612ce2293b7e39fbf87d: Status 404 returned error can't find the container with id b939620e8d61442a7745d9733db1c8e8fc90f9b1b798612ce2293b7e39fbf87d Jul 06 11:28:57.994264 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:57.993759 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:57.994264 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:57.993896 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:28:58.041460 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:58.041424 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" event={"ID":"4b09df2b-9cbd-447a-b3ee-6d3101207218","Type":"ContainerStarted","Data":"b939620e8d61442a7745d9733db1c8e8fc90f9b1b798612ce2293b7e39fbf87d"} Jul 06 11:28:58.994452 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:58.993928 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:28:58.994452 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:58.994059 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:28:58.994452 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:58.994412 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:28:58.994977 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:58.994514 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:28:59.172873 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:28:59.172594 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1068e2de_db09_452c_9b12_8ba49d8cbb33.slice/crio-bffa44e50d40ad2fe09625ebd20af1536ffe1ac2eb2118d794c795d40df1e3fc WatchSource:0}: Error finding container bffa44e50d40ad2fe09625ebd20af1536ffe1ac2eb2118d794c795d40df1e3fc: Status 404 returned error can't find the container with id bffa44e50d40ad2fe09625ebd20af1536ffe1ac2eb2118d794c795d40df1e3fc Jul 06 11:28:59.993877 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:28:59.993842 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:28:59.994049 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:28:59.993979 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:00.046531 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:00.046353 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qhh2l" event={"ID":"1068e2de-db09-452c-9b12-8ba49d8cbb33","Type":"ContainerStarted","Data":"bffa44e50d40ad2fe09625ebd20af1536ffe1ac2eb2118d794c795d40df1e3fc"} Jul 06 11:29:00.488747 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:00.488371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:00.488747 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:00.488477 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:00.488747 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.488526 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:29:00.488747 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.488605 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:29:08.48858127 +0000 UTC m=+18.020320598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:29:00.488747 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.488606 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:29:00.488747 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.488660 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:08.488644115 +0000 UTC m=+18.020383425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:29:00.690969 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:00.690719 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:00.690969 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.690868 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:29:00.690969 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.690890 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:29:00.690969 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.690903 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:29:00.690969 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.690963 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:08.690945026 +0000 UTC m=+18.222684334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:29:00.997522 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:00.997495 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:00.997697 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.997603 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:00.997765 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:00.997721 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:00.997835 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:00.997817 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:01.992948 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:01.992913 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:01.993405 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:01.993061 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:02.996449 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:02.996419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:02.996833 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:02.996424 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:02.996833 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:02.996572 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:02.996833 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:02.996667 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:03.993485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:03.993453 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:03.993624 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:03.993568 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:04.993912 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:04.993878 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:04.994370 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:04.994008 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:04.994370 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:04.994055 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:04.994370 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:04.994146 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:05.993354 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:05.993326 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:05.993546 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:05.993439 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:06.993504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:06.993473 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:06.993914 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:06.993599 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:06.993914 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:06.993657 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:06.993914 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:06.993768 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:07.993065 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:07.993035 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:07.993285 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:07.993179 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:08.552397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:08.552363 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:08.552823 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:08.552431 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:08.552823 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.552515 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:29:08.552823 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.552536 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:29:08.552823 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.552590 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:24.552568584 +0000 UTC m=+34.084307902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:29:08.552823 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.552606 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:29:24.552599341 +0000 UTC m=+34.084338645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:29:08.753516 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:08.753483 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:08.753677 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.753647 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:29:08.753677 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.753663 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:29:08.753677 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.753673 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:29:08.753844 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.753724 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:24.753711419 +0000 UTC m=+34.285450723 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:29:08.993188 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:08.993097 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:08.993333 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.993222 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:08.993333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:08.993279 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:08.993459 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:08.993400 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:09.993017 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:09.992992 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:09.993472 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:09.993097 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:10.999021 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:10.998983 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:10.999428 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:10.999084 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:10.999428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:10.999139 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:10.999428 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:10.999259 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:11.993236 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:11.992982 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:11.993388 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:11.993362 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:12.070046 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.070016 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"bb4f0bde1ee266f9fc66bcefdad21effbdd7c2ba54d18f168ec96bb8f870936b"} Jul 06 11:29:12.070679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.070099 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"f22db6398a54ce419503758308287916dc725e40a8b39a7761dcf75196eed438"} Jul 06 11:29:12.070679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.070144 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"9f88f0611f5f104231480238252a4b3b045764fc9000515e452e15e22583daf7"} Jul 06 11:29:12.070679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.070213 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"9075c1b3110b6117a3572a7a1899181c6fdd23b609db625a153b3f0464d95c60"} Jul 06 11:29:12.070679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.070254 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"da773aed5578621137b44fa6667d27b45703c60ebf9e6d39bb4d454ae970ffa3"} Jul 06 11:29:12.075014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.074823 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5qngh" event={"ID":"7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3","Type":"ContainerStarted","Data":"ceeb1810800ecb1d5d63ac8031acbfc9365404719f464b7d1e6c42c722f18da9"} Jul 06 11:29:12.076549 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.076523 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8lck9" event={"ID":"50752636-200c-4d95-ad03-1c19fdc20645","Type":"ContainerStarted","Data":"91fbdc9b1cc577d8d16d6dd6a1345b285114d75cf8e1f3c27e226dfb2773dc33"} Jul 06 11:29:12.081522 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.081485 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fsvg9" event={"ID":"b5b8fa1d-0de0-4339-bfc7-1e940522963c","Type":"ContainerStarted","Data":"e1ee9ef1deb1b719988407225f6c1e8d1ef48fb567da1057a874f746a1d67762"} Jul 06 11:29:12.084978 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.084951 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" event={"ID":"4b09df2b-9cbd-447a-b3ee-6d3101207218","Type":"ContainerStarted","Data":"3d01f733eb5ca14f36384bc42f4782545919ec51cc8f9d4e571546ccea63e174"} Jul 06 11:29:12.087722 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.087451 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" event={"ID":"fa33dbc8162804a9c2f54b7dac53c409","Type":"ContainerStarted","Data":"d1852f4b8cc61674d359c9e92c2fda2521ea72bf5503d1d5dda5ff7cb0955727"} Jul 06 11:29:12.088935 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.088912 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" event={"ID":"088e17fe59b4ec12c1114c83bdf80edd","Type":"ContainerStarted","Data":"29e7395994e2f616e968df1d2c8bee9bf315ea92db103272414f71e78328948c"} Jul 06 11:29:12.090221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.090181 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nghnk" event={"ID":"30711795-85a6-4b52-aa6b-06a1188f7b58","Type":"ContainerStarted","Data":"d36db0af63cec204cf250dccb5ddb63072aa9f8e04c254ba2cba24a443381826"} Jul 06 11:29:12.091464 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.091439 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vwx7g" event={"ID":"fc73ff93-6a36-47fb-b2f0-e85088d70097","Type":"ContainerStarted","Data":"a2eefb6299b4c444f2673b25ffe85cebc14aaf5094460e1e6c14f73d799aec72"} Jul 06 11:29:12.091766 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.091733 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8lck9" podStartSLOduration=11.707803256 podStartE2EDuration="21.091718816s" totalImagesPullingTime="9.38391556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:12.091473761 +0000 UTC m=+21.623213090" watchObservedRunningTime="2026-07-06 11:29:12.091718816 +0000 UTC m=+21.623458143" Jul 06 11:29:12.092572 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.092556 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerStarted","Data":"73039330aa9ca4f195696e8a04813946b43b9a09dd9c2b1de48ecb2115c55849"} Jul 06 11:29:12.107317 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.107285 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fsvg9" podStartSLOduration=2.753496826 podStartE2EDuration="21.107274231s" totalImagesPullingTime="18.353777405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:12.107106797 +0000 UTC m=+21.638846124" watchObservedRunningTime="2026-07-06 11:29:12.107274231 +0000 UTC m=+21.639013557" Jul 06 11:29:12.142056 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.141861 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-173.ec2.internal" podStartSLOduration=20.141845619 podStartE2EDuration="20.141845619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:12.141486231 +0000 UTC m=+21.673225561" watchObservedRunningTime="2026-07-06 11:29:12.141845619 +0000 UTC m=+21.673584944" Jul 06 11:29:12.182475 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.182440 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vwx7g" podStartSLOduration=2.761134138 podStartE2EDuration="21.182427919s" totalImagesPullingTime="18.421293781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:12.16379437 +0000 UTC m=+21.695533717" watchObservedRunningTime="2026-07-06 11:29:12.182427919 +0000 UTC m=+21.714167244" Jul 06 11:29:12.182640 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.182615 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nghnk" podStartSLOduration=2.835386288 podStartE2EDuration="21.182611295s" totalImagesPullingTime="18.347225007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:12.181449112 +0000 UTC m=+21.713188437" watchObservedRunningTime="2026-07-06 11:29:12.182611295 +0000 UTC m=+21.714350618" Jul 06 11:29:12.227723 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.227689 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:12.297897 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.297867 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:12.365972 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.365948 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:12.440814 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.440740 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:12.993583 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.993546 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:12.993720 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:12.993692 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:12.993765 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:12.993719 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:12.993816 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:12.993799 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:13.096362 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.096303 2585 generic.go:356] "Generic (PLEG): container finished" podID="fa33dbc8162804a9c2f54b7dac53c409" containerID="d1852f4b8cc61674d359c9e92c2fda2521ea72bf5503d1d5dda5ff7cb0955727" exitCode=0 Jul 06 11:29:13.096362 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.096331 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" event={"ID":"fa33dbc8162804a9c2f54b7dac53c409","Type":"ContainerDied","Data":"d1852f4b8cc61674d359c9e92c2fda2521ea72bf5503d1d5dda5ff7cb0955727"} Jul 06 11:29:13.096766 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.096369 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" event={"ID":"fa33dbc8162804a9c2f54b7dac53c409","Type":"ContainerStarted","Data":"cb92b6f8e4c54ae0c360bdab0ac735e2ecae85e87d746351dc79f11f4505afe0"} Jul 06 11:29:13.100345 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.100322 2585 generic.go:356] "Generic (PLEG): container finished" podID="49de6972-f0b8-4bc0-a83d-b837a63e3a6f" containerID="73039330aa9ca4f195696e8a04813946b43b9a09dd9c2b1de48ecb2115c55849" exitCode=0 Jul 06 11:29:13.100482 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.100385 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerDied","Data":"73039330aa9ca4f195696e8a04813946b43b9a09dd9c2b1de48ecb2115c55849"} Jul 06 11:29:13.101818 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.101797 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qhh2l" event={"ID":"1068e2de-db09-452c-9b12-8ba49d8cbb33","Type":"ContainerStarted","Data":"50fe98f4a4865e4426781d997f28076e23929b813409dfbd050cf23b0b16b1d8"} Jul 06 11:29:13.104976 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.104949 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"4bfa82a9a9a0c47840534f2f881dbcd4f52f9cb7b378dd5094e5ca32ab7dce49"} Jul 06 11:29:13.116464 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.116428 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-173.ec2.internal" podStartSLOduration=21.116416733 podStartE2EDuration="21.116416733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:13.116124509 +0000 UTC m=+22.647863847" watchObservedRunningTime="2026-07-06 11:29:13.116416733 +0000 UTC m=+22.648156058" Jul 06 11:29:13.131213 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.131180 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5qngh" podStartSLOduration=3.786940896 podStartE2EDuration="22.131149547s" totalImagesPullingTime="18.344208651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:13.130948669 +0000 UTC m=+22.662687994" watchObservedRunningTime="2026-07-06 11:29:13.131149547 +0000 UTC m=+22.662888874" Jul 06 11:29:13.173920 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.173890 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qhh2l" podStartSLOduration=22.173874 podStartE2EDuration="22.173874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:13.173469451 +0000 UTC m=+22.705208777" watchObservedRunningTime="2026-07-06 11:29:13.173874 +0000 UTC m=+22.705613327" Jul 06 11:29:13.206992 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.206969 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:13.300568 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.300543 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:13.380012 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.379993 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 11:29:13.405844 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.405822 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:13.873191 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.873093 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T11:29:13.380008734Z","UUID":"bc038314-80e5-4385-b18a-9df622700910","Handler":null,"Name":"","Endpoint":""} Jul 06 11:29:13.874643 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.874627 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 11:29:13.874732 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.874660 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 11:29:13.924846 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.924821 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:29:13.925508 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.925467 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:29:13.993601 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:13.993582 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:13.993720 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:13.993699 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:14.071603 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.071577 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:14.109008 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.108980 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" event={"ID":"4b09df2b-9cbd-447a-b3ee-6d3101207218","Type":"ContainerStarted","Data":"e0b221230f11e6db1835a6b5f5aa2860577a0bf7be63bd59e4b26154de690d12"} Jul 06 11:29:14.109824 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.109782 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:29:14.110385 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.110370 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fsvg9" Jul 06 11:29:14.187368 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.187250 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:14.993727 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.993695 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:14.993897 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:14.993858 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:14.993973 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:14.993899 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:14.994029 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:14.993981 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:15.114346 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:15.114316 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"7d2c92958594ed4929f86393047e5664524cb3073c8a99d752731c361694de4d"} Jul 06 11:29:15.116125 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:15.116095 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" event={"ID":"4b09df2b-9cbd-447a-b3ee-6d3101207218","Type":"ContainerStarted","Data":"98407b3c321678887e9b62ed80c979c3aaeaa7d44d31f9fd9565dda0bde2357d"} Jul 06 11:29:15.137173 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:15.137131 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2p5n" podStartSLOduration=8.317716658 podStartE2EDuration="24.13711936s" totalImagesPullingTime="15.819402702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:15.136548294 +0000 UTC m=+24.668287622" watchObservedRunningTime="2026-07-06 11:29:15.13711936 +0000 UTC m=+24.668858697" Jul 06 11:29:15.993408 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:15.993380 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:15.993571 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:15.993504 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:16.993783 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:16.993748 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:16.994187 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:16.993797 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:16.994187 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:16.993884 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:16.994187 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:16.994026 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:17.993788 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:17.993758 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:17.994133 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:17.993863 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:18.993549 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:18.993367 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:18.993661 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:18.993434 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:18.993716 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:18.993652 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:18.993716 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:18.993695 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:19.129007 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.128945 2585 generic.go:356] "Generic (PLEG): container finished" podID="49de6972-f0b8-4bc0-a83d-b837a63e3a6f" containerID="c4d934ad2d306a0067209dcc2023d8498ef5a012f4b99add3253a1a2c5b843d4" exitCode=0 Jul 06 11:29:19.129799 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.129033 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerDied","Data":"c4d934ad2d306a0067209dcc2023d8498ef5a012f4b99add3253a1a2c5b843d4"} Jul 06 11:29:19.133860 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.133835 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" event={"ID":"48fea88c-3651-4ef7-9403-3dc063c0468d","Type":"ContainerStarted","Data":"d5dac985ba94b439ec3981e9efd79576db71c0a031a67548b42cbadd5b52c607"} Jul 06 11:29:19.134145 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.134110 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:29:19.134276 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.134218 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:29:19.150624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.150604 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:29:19.150893 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.150876 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:29:19.186565 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.186522 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" podStartSLOduration=9.693890901 podStartE2EDuration="28.186508792s" totalImagesPullingTime="18.492617891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:19.186199771 +0000 UTC m=+28.717939098" watchObservedRunningTime="2026-07-06 11:29:19.186508792 +0000 UTC m=+28.718248117" Jul 06 11:29:19.690554 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.690526 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:19.792191 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.792143 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:19.993552 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:19.993493 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:19.993681 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:19.993593 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:20.136681 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.136654 2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 11:29:20.365728 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.365571 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q8q7d"] Jul 06 11:29:20.365854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.365841 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:20.365951 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:20.365931 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:20.368765 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.368745 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pfkrn"] Jul 06 11:29:20.368847 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.368832 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:20.368932 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:20.368911 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:20.369255 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.369234 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-66m6t"] Jul 06 11:29:20.369332 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:20.369319 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:20.369413 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:20.369396 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:21.140474 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.140439 2585 generic.go:356] "Generic (PLEG): container finished" podID="49de6972-f0b8-4bc0-a83d-b837a63e3a6f" containerID="46fb535f3cde8c5ff7dd7fbe4b483bb984b1f347d49261ae3cae6947a0780500" exitCode=0 Jul 06 11:29:21.140768 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.140526 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerDied","Data":"46fb535f3cde8c5ff7dd7fbe4b483bb984b1f347d49261ae3cae6947a0780500"} Jul 06 11:29:21.140768 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.140589 2585 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 11:29:21.149439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.149419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:21.253189 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.253124 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:21.460258 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.460242 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:29:21.993480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.993449 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:21.993662 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.993564 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:21.993662 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:21.993570 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:21.993662 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:21.993592 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:21.993839 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:21.993668 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:21.993839 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:21.993768 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:22.953485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:22.953422 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:23.054805 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.054777 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:23.146933 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.146907 2585 generic.go:356] "Generic (PLEG): container finished" podID="49de6972-f0b8-4bc0-a83d-b837a63e3a6f" containerID="6fe1a393e83b7710ba46ba64c6a982578744c8065091f138290d2322a690dda9" exitCode=0 Jul 06 11:29:23.147018 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.146939 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerDied","Data":"6fe1a393e83b7710ba46ba64c6a982578744c8065091f138290d2322a690dda9"} Jul 06 11:29:23.378735 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.378682 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:23.478915 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.478892 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:23.993811 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.993783 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:23.994200 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.993886 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:23.994200 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:23.993900 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-66m6t" podUID="1f8a3be6-208b-46d8-a37e-4e058d9d0613" Jul 06 11:29:23.994200 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:23.993913 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:23.994200 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:23.993977 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q8q7d" podUID="5a6ad56c-e987-47b6-a64c-acf6705a3228" Jul 06 11:29:23.994200 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:23.994065 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfkrn" podUID="672720fa-209d-4354-8698-27024f8e88ec" Jul 06 11:29:24.237883 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.237858 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-173.ec2.internal" event="NodeReady" Jul 06 11:29:24.237996 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.237965 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 11:29:24.291574 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.291462 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-467xf"] Jul 06 11:29:24.304515 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.304488 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ss752"] Jul 06 11:29:24.304640 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.304614 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.306743 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.306698 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bpbrx\"" Jul 06 11:29:24.307082 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.306938 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 11:29:24.307082 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.306969 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 11:29:24.319320 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.319297 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-467xf"] Jul 06 11:29:24.319428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.319331 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ss752"] Jul 06 11:29:24.319428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.319415 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.321644 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.321620 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 11:29:24.321749 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.321641 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 11:29:24.321749 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.321686 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 11:29:24.322300 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.322113 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-989xb\"" Jul 06 11:29:24.412450 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.412419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:24.468372 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.468347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.468480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.468377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bf26ed81-48f0-4641-8eb3-530e8699cc52-config-volume\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.468480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.468405 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zlgv\" (UniqueName: \"kubernetes.io/projected/bf26ed81-48f0-4641-8eb3-530e8699cc52-kube-api-access-8zlgv\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.468567 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.468492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbbd9\" (UniqueName: \"kubernetes.io/projected/d44c25d8-3db3-400e-a94e-5b6a34b3c0f5-kube-api-access-pbbd9\") pod \"ingress-canary-ss752\" (UID: \"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5\") " pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.468567 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.468520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf26ed81-48f0-4641-8eb3-530e8699cc52-tmp-dir\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.468664 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.468588 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d44c25d8-3db3-400e-a94e-5b6a34b3c0f5-cert\") pod \"ingress-canary-ss752\" (UID: \"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5\") " pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.569755 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569698 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:24.569755 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d44c25d8-3db3-400e-a94e-5b6a34b3c0f5-cert\") pod \"ingress-canary-ss752\" (UID: \"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5\") " pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.569880 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569759 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.569880 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569811 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bf26ed81-48f0-4641-8eb3-530e8699cc52-config-volume\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.569880 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zlgv\" (UniqueName: \"kubernetes.io/projected/bf26ed81-48f0-4641-8eb3-530e8699cc52-kube-api-access-8zlgv\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.569880 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbbd9\" (UniqueName: \"kubernetes.io/projected/d44c25d8-3db3-400e-a94e-5b6a34b3c0f5-kube-api-access-pbbd9\") pod \"ingress-canary-ss752\" (UID: \"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5\") " pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.570046 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569900 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf26ed81-48f0-4641-8eb3-530e8699cc52-tmp-dir\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.570046 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.569840 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 11:29:24.570046 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.569920 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:24.570046 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.569972 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls podName:bf26ed81-48f0-4641-8eb3-530e8699cc52 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:25.069952007 +0000 UTC m=+34.601691310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls") pod "dns-default-467xf" (UID: "bf26ed81-48f0-4641-8eb3-530e8699cc52") : secret "dns-default-metrics-tls" not found Jul 06 11:29:24.570265 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.569840 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 11:29:24.570265 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.570068 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:29:24.570265 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.570115 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret podName:5a6ad56c-e987-47b6-a64c-acf6705a3228 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:56.570098604 +0000 UTC m=+66.101837920 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret") pod "global-pull-secret-syncer-q8q7d" (UID: "5a6ad56c-e987-47b6-a64c-acf6705a3228") : object "kube-system"/"original-pull-secret" not registered Jul 06 11:29:24.570265 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.570130 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs podName:672720fa-209d-4354-8698-27024f8e88ec nodeName:}" failed. No retries permitted until 2026-07-06 11:29:56.570122921 +0000 UTC m=+66.101862227 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs") pod "network-metrics-daemon-pfkrn" (UID: "672720fa-209d-4354-8698-27024f8e88ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 11:29:24.570408 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.570259 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf26ed81-48f0-4641-8eb3-530e8699cc52-tmp-dir\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.570408 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.570393 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bf26ed81-48f0-4641-8eb3-530e8699cc52-config-volume\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.573817 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.573788 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d44c25d8-3db3-400e-a94e-5b6a34b3c0f5-cert\") pod \"ingress-canary-ss752\" (UID: \"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5\") " pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.578552 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.578532 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zlgv\" (UniqueName: \"kubernetes.io/projected/bf26ed81-48f0-4641-8eb3-530e8699cc52-kube-api-access-8zlgv\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:24.578640 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.578601 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbbd9\" (UniqueName: \"kubernetes.io/projected/d44c25d8-3db3-400e-a94e-5b6a34b3c0f5-kube-api-access-pbbd9\") pod \"ingress-canary-ss752\" (UID: \"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5\") " pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.633211 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.633179 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ss752" Jul 06 11:29:24.771186 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.770982 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:24.771314 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.771144 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 11:29:24.771358 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.771348 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 11:29:24.771434 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.771370 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2skvc for pod openshift-network-diagnostics/network-check-target-66m6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:29:24.771488 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:24.771463 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc podName:1f8a3be6-208b-46d8-a37e-4e058d9d0613 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:56.77143677 +0000 UTC m=+66.303176078 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2skvc" (UniqueName: "kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc") pod "network-check-target-66m6t" (UID: "1f8a3be6-208b-46d8-a37e-4e058d9d0613") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 11:29:24.801211 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:24.801185 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ss752"] Jul 06 11:29:24.805891 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:24.805862 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44c25d8_3db3_400e_a94e_5b6a34b3c0f5.slice/crio-46f6f5385ee9c44a1a0ca67f86633a0ef8f84f0b5c21efddb8644e39caee9dfd WatchSource:0}: Error finding container 46f6f5385ee9c44a1a0ca67f86633a0ef8f84f0b5c21efddb8644e39caee9dfd: Status 404 returned error can't find the container with id 46f6f5385ee9c44a1a0ca67f86633a0ef8f84f0b5c21efddb8644e39caee9dfd Jul 06 11:29:25.074263 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.074236 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:25.074905 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:25.074341 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 11:29:25.074905 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:25.074433 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls podName:bf26ed81-48f0-4641-8eb3-530e8699cc52 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:26.074414941 +0000 UTC m=+35.606154259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls") pod "dns-default-467xf" (UID: "bf26ed81-48f0-4641-8eb3-530e8699cc52") : secret "dns-default-metrics-tls" not found Jul 06 11:29:25.153766 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.153735 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ss752" event={"ID":"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5","Type":"ContainerStarted","Data":"46f6f5385ee9c44a1a0ca67f86633a0ef8f84f0b5c21efddb8644e39caee9dfd"} Jul 06 11:29:25.676897 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.676867 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xwdrh"] Jul 06 11:29:25.697945 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.697913 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xwdrh"] Jul 06 11:29:25.698097 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.698040 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.700945 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.700491 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 11:29:25.700945 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.700512 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hcchr\"" Jul 06 11:29:25.700945 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.700643 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 11:29:25.700945 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.700726 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 11:29:25.700945 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.700818 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 11:29:25.714083 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.714059 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:25.773939 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.773913 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:25.881369 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.881338 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.881502 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.881376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9nd\" (UniqueName: \"kubernetes.io/projected/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-api-access-qh9nd\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.881502 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.881414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.881502 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.881487 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8052bc97-28d9-466e-87fa-5435fbd1baa1-crio-socket\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.881671 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.881518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8052bc97-28d9-466e-87fa-5435fbd1baa1-data-volume\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.982890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.982821 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.982890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.982861 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9nd\" (UniqueName: \"kubernetes.io/projected/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-api-access-qh9nd\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.983055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.982894 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.983055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.982923 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8052bc97-28d9-466e-87fa-5435fbd1baa1-crio-socket\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.983055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.982952 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8052bc97-28d9-466e-87fa-5435fbd1baa1-data-volume\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.983055 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:25.982980 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 11:29:25.983055 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:25.983041 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:26.483022081 +0000 UTC m=+36.014761402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : secret "insights-runtime-extractor-tls" not found Jul 06 11:29:25.983055 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:25.983047 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 11:29:25.983358 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:25.983100 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:26.48308747 +0000 UTC m=+36.014826775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : configmap "kube-rbac-proxy" not found Jul 06 11:29:25.983358 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.983232 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8052bc97-28d9-466e-87fa-5435fbd1baa1-crio-socket\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.983358 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.983311 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8052bc97-28d9-466e-87fa-5435fbd1baa1-data-volume\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.993437 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.993386 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:25.993437 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.993403 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:25.993599 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.993490 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:25.993740 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.993707 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9nd\" (UniqueName: \"kubernetes.io/projected/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-api-access-qh9nd\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:25.995518 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.995496 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kdds4\"" Jul 06 11:29:25.995839 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.995813 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 11:29:25.995939 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.995843 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 11:29:25.996039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.995969 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 11:29:25.996232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.996203 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 11:29:25.996301 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:25.996239 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w64mh\"" Jul 06 11:29:26.084116 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.084088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:26.084668 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:26.084261 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 11:29:26.084668 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:26.084333 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls podName:bf26ed81-48f0-4641-8eb3-530e8699cc52 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:28.084313511 +0000 UTC m=+37.616052834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls") pod "dns-default-467xf" (UID: "bf26ed81-48f0-4641-8eb3-530e8699cc52") : secret "dns-default-metrics-tls" not found Jul 06 11:29:26.214490 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.214456 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:26.304083 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.304016 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:26.407242 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.407081 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:26.488331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.488303 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:26.488490 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.488348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:26.488490 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:26.488433 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 11:29:26.488490 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:26.488477 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 11:29:26.488649 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:26.488497 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:27.48847961 +0000 UTC m=+37.020218918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : configmap "kube-rbac-proxy" not found Jul 06 11:29:26.488649 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:26.488530 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:27.488513196 +0000 UTC m=+37.020252506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : secret "insights-runtime-extractor-tls" not found Jul 06 11:29:26.532492 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.532462 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:26.549763 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.549740 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp"] Jul 06 11:29:26.568195 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.568123 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" Jul 06 11:29:26.571099 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.570929 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 11:29:26.571292 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.571276 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 11:29:26.571593 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.571574 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dblvq\"" Jul 06 11:29:26.571889 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.571857 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp"] Jul 06 11:29:26.612970 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.612943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:26.686077 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.686048 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:26.690031 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.690008 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26vbl\" (UniqueName: \"kubernetes.io/projected/c7fd9b79-a187-47ae-8cb6-ce5a46152687-kube-api-access-26vbl\") pod \"migrator-d57d58cc9-6whvp\" (UID: \"c7fd9b79-a187-47ae-8cb6-ce5a46152687\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" Jul 06 11:29:26.790933 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.790901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26vbl\" (UniqueName: \"kubernetes.io/projected/c7fd9b79-a187-47ae-8cb6-ce5a46152687-kube-api-access-26vbl\") pod \"migrator-d57d58cc9-6whvp\" (UID: \"c7fd9b79-a187-47ae-8cb6-ce5a46152687\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" Jul 06 11:29:26.805951 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.805924 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26vbl\" (UniqueName: \"kubernetes.io/projected/c7fd9b79-a187-47ae-8cb6-ce5a46152687-kube-api-access-26vbl\") pod \"migrator-d57d58cc9-6whvp\" (UID: \"c7fd9b79-a187-47ae-8cb6-ce5a46152687\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" Jul 06 11:29:26.879969 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:26.879911 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" Jul 06 11:29:27.025914 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.025882 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:27.114914 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.114880 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:27.226099 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.226042 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:27.305542 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.305514 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:27.464303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.464278 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8lck9_50752636-200c-4d95-ad03-1c19fdc20645/dns-node-resolver/0.log" Jul 06 11:29:27.495577 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.495501 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:27.495577 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.495551 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:27.495752 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:27.495628 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 11:29:27.495752 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:27.495649 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 11:29:27.495752 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:27.495692 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:29.495673555 +0000 UTC m=+39.027412858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : configmap "kube-rbac-proxy" not found Jul 06 11:29:27.495752 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:27.495705 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:29.495699239 +0000 UTC m=+39.027438543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : secret "insights-runtime-extractor-tls" not found Jul 06 11:29:27.814109 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.814029 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm"] Jul 06 11:29:27.826904 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.826885 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" Jul 06 11:29:27.830432 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.830413 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f9bm5\"" Jul 06 11:29:27.840039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.840017 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm"] Jul 06 11:29:27.998353 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:27.998311 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkb7g\" (UniqueName: \"kubernetes.io/projected/de5e3792-d557-479f-9e94-ef1cf77bca02-kube-api-access-jkb7g\") pod \"network-check-source-989f8b8dc-t4dxm\" (UID: \"de5e3792-d557-479f-9e94-ef1cf77bca02\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" Jul 06 11:29:28.099702 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.099626 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:28.099834 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:28.099800 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 11:29:28.099884 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.099841 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkb7g\" (UniqueName: \"kubernetes.io/projected/de5e3792-d557-479f-9e94-ef1cf77bca02-kube-api-access-jkb7g\") pod \"network-check-source-989f8b8dc-t4dxm\" (UID: \"de5e3792-d557-479f-9e94-ef1cf77bca02\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" Jul 06 11:29:28.099884 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:28.099873 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls podName:bf26ed81-48f0-4641-8eb3-530e8699cc52 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:32.09984625 +0000 UTC m=+41.631585568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls") pod "dns-default-467xf" (UID: "bf26ed81-48f0-4641-8eb3-530e8699cc52") : secret "dns-default-metrics-tls" not found Jul 06 11:29:28.122941 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.122919 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkb7g\" (UniqueName: \"kubernetes.io/projected/de5e3792-d557-479f-9e94-ef1cf77bca02-kube-api-access-jkb7g\") pod \"network-check-source-989f8b8dc-t4dxm\" (UID: \"de5e3792-d557-479f-9e94-ef1cf77bca02\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" Jul 06 11:29:28.136595 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.136563 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" Jul 06 11:29:28.469147 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.469067 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5qngh_7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3/node-ca/0.log" Jul 06 11:29:28.610194 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.609253 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-jw4h6"] Jul 06 11:29:28.617918 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.617291 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.619872 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.619846 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6qfvv\"" Jul 06 11:29:28.620118 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.620102 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 11:29:28.620455 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.620438 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 11:29:28.620880 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.620655 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 11:29:28.620880 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.620727 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 11:29:28.623965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.623943 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-jw4h6"] Jul 06 11:29:28.645539 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.645501 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:28.666646 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.666622 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp"] Jul 06 11:29:28.671420 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:28.671397 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7fd9b79_a187_47ae_8cb6_ce5a46152687.slice/crio-f45bc1d48db4358784a4e8166b8aa56f9aead3c663b79baba3a813d2d7ed4e33 WatchSource:0}: Error finding container f45bc1d48db4358784a4e8166b8aa56f9aead3c663b79baba3a813d2d7ed4e33: Status 404 returned error can't find the container with id f45bc1d48db4358784a4e8166b8aa56f9aead3c663b79baba3a813d2d7ed4e33 Jul 06 11:29:28.679384 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.679358 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm"] Jul 06 11:29:28.682219 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:28.682191 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5e3792_d557_479f_9e94_ef1cf77bca02.slice/crio-5003a9286450e4f51a4be92ae1e4669b8b1dbbce41e47160fec757b6f49692fc WatchSource:0}: Error finding container 5003a9286450e4f51a4be92ae1e4669b8b1dbbce41e47160fec757b6f49692fc: Status 404 returned error can't find the container with id 5003a9286450e4f51a4be92ae1e4669b8b1dbbce41e47160fec757b6f49692fc Jul 06 11:29:28.805390 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.805358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-signing-cabundle\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.805499 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.805393 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-signing-key\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.805499 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.805423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvdgk\" (UniqueName: \"kubernetes.io/projected/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-kube-api-access-pvdgk\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.906476 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.906449 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-signing-cabundle\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.906580 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.906480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-signing-key\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.906580 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.906509 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvdgk\" (UniqueName: \"kubernetes.io/projected/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-kube-api-access-pvdgk\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.907174 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.907140 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-signing-cabundle\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.909775 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.909755 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-signing-key\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.914057 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.914039 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvdgk\" (UniqueName: \"kubernetes.io/projected/13abff8f-2b5f-4e2f-8bf6-e19ca95739b6-kube-api-access-pvdgk\") pod \"service-ca-758f599c5-jw4h6\" (UID: \"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6\") " pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:28.941034 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:28.941014 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-jw4h6" Jul 06 11:29:29.094420 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.094394 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-jw4h6"] Jul 06 11:29:29.097544 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:29.097518 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13abff8f_2b5f_4e2f_8bf6_e19ca95739b6.slice/crio-a4faf91a84ff36c89ad450534d26ac3fb5e6830faa6072bc0262c225dad7fe98 WatchSource:0}: Error finding container a4faf91a84ff36c89ad450534d26ac3fb5e6830faa6072bc0262c225dad7fe98: Status 404 returned error can't find the container with id a4faf91a84ff36c89ad450534d26ac3fb5e6830faa6072bc0262c225dad7fe98 Jul 06 11:29:29.167779 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.167616 2585 generic.go:356] "Generic (PLEG): container finished" podID="49de6972-f0b8-4bc0-a83d-b837a63e3a6f" containerID="8ff321b77dfe9e74dbde694e64914b290f45c9134aebddda9d15d72ea800eb4c" exitCode=0 Jul 06 11:29:29.168149 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.167680 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerDied","Data":"8ff321b77dfe9e74dbde694e64914b290f45c9134aebddda9d15d72ea800eb4c"} Jul 06 11:29:29.169411 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.169363 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-jw4h6" event={"ID":"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6","Type":"ContainerStarted","Data":"a4faf91a84ff36c89ad450534d26ac3fb5e6830faa6072bc0262c225dad7fe98"} Jul 06 11:29:29.170774 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.170750 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" event={"ID":"c7fd9b79-a187-47ae-8cb6-ce5a46152687","Type":"ContainerStarted","Data":"f45bc1d48db4358784a4e8166b8aa56f9aead3c663b79baba3a813d2d7ed4e33"} Jul 06 11:29:29.172435 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.172319 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ss752" event={"ID":"d44c25d8-3db3-400e-a94e-5b6a34b3c0f5","Type":"ContainerStarted","Data":"224fed18825f993484c64889246bc472e0e3f9ddbe92a657c21fdbe4058993f6"} Jul 06 11:29:29.176944 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.174372 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" event={"ID":"de5e3792-d557-479f-9e94-ef1cf77bca02","Type":"ContainerStarted","Data":"5003a9286450e4f51a4be92ae1e4669b8b1dbbce41e47160fec757b6f49692fc"} Jul 06 11:29:29.230954 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.230906 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ss752" podStartSLOduration=1.505177413 podStartE2EDuration="5.230890503s" totalImagesPullingTime="3.72571309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:29.230091141 +0000 UTC m=+38.761830467" watchObservedRunningTime="2026-07-06 11:29:29.230890503 +0000 UTC m=+38.762629831" Jul 06 11:29:29.465818 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.465792 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ss752_d44c25d8-3db3-400e-a94e-5b6a34b3c0f5/serve-healthcheck-canary/0.log" Jul 06 11:29:29.513118 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.513089 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:29.513256 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.513145 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:29.513256 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:29.513216 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 11:29:29.513256 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:29.513223 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 11:29:29.513429 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:29.513277 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:33.513258971 +0000 UTC m=+43.044998276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : secret "insights-runtime-extractor-tls" not found Jul 06 11:29:29.513429 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:29.513296 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:33.513286394 +0000 UTC m=+43.045025702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : configmap "kube-rbac-proxy" not found Jul 06 11:29:29.568153 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:29.568120 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:30.182192 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:30.181706 2585 generic.go:356] "Generic (PLEG): container finished" podID="49de6972-f0b8-4bc0-a83d-b837a63e3a6f" containerID="5c9171ba9aab5c2b8679f324db4efe6e1c65f1ca41008eae94a31e77263121d9" exitCode=0 Jul 06 11:29:30.182192 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:30.181849 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerDied","Data":"5c9171ba9aab5c2b8679f324db4efe6e1c65f1ca41008eae94a31e77263121d9"} Jul 06 11:29:30.316444 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:30.316421 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:31.192241 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:31.192198 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" event={"ID":"49de6972-f0b8-4bc0-a83d-b837a63e3a6f","Type":"ContainerStarted","Data":"3b888ee0f7f8e71d5d9446bd48cf9e0af2f106f29ae717815cc59d9c5d0b6056"} Jul 06 11:29:31.194263 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:31.194230 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" event={"ID":"c7fd9b79-a187-47ae-8cb6-ce5a46152687","Type":"ContainerStarted","Data":"ccafd26e6b7fc7ed078f3372b98dbed2373482334b5bd6d8e9e3e09cec733b29"} Jul 06 11:29:31.194397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:31.194282 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" event={"ID":"c7fd9b79-a187-47ae-8cb6-ce5a46152687","Type":"ContainerStarted","Data":"cdfd6644ff986543bb2a3ed7474dd28cfcfce9ab09e7bc28bd79633db24ca89a"} Jul 06 11:29:31.220665 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:31.220619 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8n4tf" podStartSLOduration=9.0152856 podStartE2EDuration="40.22060215s" totalImagesPullingTime="31.20531655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:31.219522761 +0000 UTC m=+40.751262088" watchObservedRunningTime="2026-07-06 11:29:31.22060215 +0000 UTC m=+40.752341479" Jul 06 11:29:31.237413 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:31.237368 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6whvp" podStartSLOduration=3.474999352 podStartE2EDuration="5.237350375s" totalImagesPullingTime="1.762351023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:31.236172745 +0000 UTC m=+40.767912061" watchObservedRunningTime="2026-07-06 11:29:31.237350375 +0000 UTC m=+40.769089700" Jul 06 11:29:32.133649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:32.133611 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:32.133831 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:32.133734 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 11:29:32.133831 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:32.133809 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls podName:bf26ed81-48f0-4641-8eb3-530e8699cc52 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:40.133790808 +0000 UTC m=+49.665530127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls") pod "dns-default-467xf" (UID: "bf26ed81-48f0-4641-8eb3-530e8699cc52") : secret "dns-default-metrics-tls" not found Jul 06 11:29:33.200286 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:33.200251 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" event={"ID":"de5e3792-d557-479f-9e94-ef1cf77bca02","Type":"ContainerStarted","Data":"76981939586f43b2ea35e6c0991a5a22ea5a8370dce291fc5290d6ed021d784b"} Jul 06 11:29:33.201742 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:33.201721 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-jw4h6" event={"ID":"13abff8f-2b5f-4e2f-8bf6-e19ca95739b6","Type":"ContainerStarted","Data":"38b10f1f7b7d9a39d3fa4ea310ecde6355b0f56e86d8fd47d0e786bd2b5b09cc"} Jul 06 11:29:33.222954 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:33.222910 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t4dxm" podStartSLOduration=2.213368512 podStartE2EDuration="6.22289714s" totalImagesPullingTime="4.009528628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:33.222622104 +0000 UTC m=+42.754361431" watchObservedRunningTime="2026-07-06 11:29:33.22289714 +0000 UTC m=+42.754636466" Jul 06 11:29:33.238688 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:33.238650 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-jw4h6" podStartSLOduration=1.63898009 podStartE2EDuration="5.238636452s" totalImagesPullingTime="3.599656362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:33.237610256 +0000 UTC m=+42.769349583" watchObservedRunningTime="2026-07-06 11:29:33.238636452 +0000 UTC m=+42.770375777" Jul 06 11:29:33.541581 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:33.541512 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:33.541694 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:33.541610 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:33.541694 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:33.541650 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 11:29:33.541762 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:33.541701 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 11:29:33.541762 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:33.541714 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:41.541696492 +0000 UTC m=+51.073435809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : configmap "kube-rbac-proxy" not found Jul 06 11:29:33.541762 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:33.541737 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:41.541727487 +0000 UTC m=+51.073466808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : secret "insights-runtime-extractor-tls" not found Jul 06 11:29:34.139061 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:34.139018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:34.414479 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:34.414395 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:34.514346 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:34.514318 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:34.614482 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:34.614432 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:34.731195 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:34.731091 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:34.837789 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:34.837758 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:35.837487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:35.837453 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:36.345663 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:36.345623 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:36.462198 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:36.462155 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:36.642906 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:36.642847 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:36.829992 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:36.829967 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:37.635111 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:37.635080 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:37.831926 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:37.831901 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:37.988549 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:37.988493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:38.086575 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:38.086550 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:38.229703 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:38.229681 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:38.837975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:38.837952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:38.939119 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:38.939092 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:39.449807 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:39.449781 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:39.528344 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:39.528322 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:39.641990 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:39.641965 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:39.841846 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:39.841823 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:40.056529 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:40.056505 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:40.190906 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:40.190853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:40.193921 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:40.193897 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf26ed81-48f0-4641-8eb3-530e8699cc52-metrics-tls\") pod \"dns-default-467xf\" (UID: \"bf26ed81-48f0-4641-8eb3-530e8699cc52\") " pod="openshift-dns/dns-default-467xf" Jul 06 11:29:40.216764 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:40.216746 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-467xf" Jul 06 11:29:40.337992 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:40.337964 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-467xf"] Jul 06 11:29:40.478148 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:40.478091 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:41.075999 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.075971 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:41.162756 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.162725 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:41.224072 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.224040 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-467xf" event={"ID":"bf26ed81-48f0-4641-8eb3-530e8699cc52","Type":"ContainerStarted","Data":"a5e2b1ffb2422e02be48fc6029c4ec3d8e8f0c1520ebd91a9e270e96ec66d8aa"} Jul 06 11:29:41.437401 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.437294 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:41.601257 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.601232 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:41.601397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.601322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:41.601446 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:41.601411 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 11:29:41.601490 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:41.601464 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls podName:8052bc97-28d9-466e-87fa-5435fbd1baa1 nodeName:}" failed. No retries permitted until 2026-07-06 11:29:57.601448188 +0000 UTC m=+67.133187497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xwdrh" (UID: "8052bc97-28d9-466e-87fa-5435fbd1baa1") : secret "insights-runtime-extractor-tls" not found Jul 06 11:29:41.601756 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.601737 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8052bc97-28d9-466e-87fa-5435fbd1baa1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:41.637039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.637010 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:41.856971 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:41.856946 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:42.246951 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:42.246929 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:43.039883 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.039846 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:43.231846 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.231816 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-467xf" event={"ID":"bf26ed81-48f0-4641-8eb3-530e8699cc52","Type":"ContainerStarted","Data":"63958b426a1b0571d61411c4987c8751de704286e5ac2cadc0f819b908d92258"} Jul 06 11:29:43.231975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.231868 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-467xf" event={"ID":"bf26ed81-48f0-4641-8eb3-530e8699cc52","Type":"ContainerStarted","Data":"d82a36a7bd6bf1a7ad5a024a4b5dcbdef30ed5a6116af48d6f13b23db6cdcd02"} Jul 06 11:29:43.231975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.231962 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-467xf" Jul 06 11:29:43.260096 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.260055 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-467xf" podStartSLOduration=17.315621088 podStartE2EDuration="19.260042291s" totalImagesPullingTime="1.944421203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:43.258895376 +0000 UTC m=+52.790634704" watchObservedRunningTime="2026-07-06 11:29:43.260042291 +0000 UTC m=+52.791781617" Jul 06 11:29:43.261568 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.261549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:43.479521 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.479469 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:43.837130 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:43.837101 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:44.432975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:44.432946 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:44.534423 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:44.534398 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:44.634544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:44.634516 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:44.703854 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:44.703807 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:44.822560 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:44.822537 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:44.887124 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:44.887101 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:45.201366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:45.201343 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:45.430852 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:45.430826 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:46.005247 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:46.005217 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:46.198277 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:46.198249 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:46.738798 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:46.738766 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:50.906140 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:50.906108 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:52.153454 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:52.153425 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bcrwd" Jul 06 11:29:53.237301 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:53.237267 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-467xf" Jul 06 11:29:54.225438 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:54.225406 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:54.326488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:54.326469 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:54.426199 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:54.426175 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:54.493531 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:54.493469 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:55.223975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:55.223951 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:55.326595 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:55.326571 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:55.427189 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:55.427150 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:55.634621 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:55.634599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:55.733835 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:55.733811 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:56.225959 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.225931 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:56.290231 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.290208 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:56.601033 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.601007 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:56.601342 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.601051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:56.603750 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.603736 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 11:29:56.604452 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.604440 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 11:29:56.613378 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.613353 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a6ad56c-e987-47b6-a64c-acf6705a3228-original-pull-secret\") pod \"global-pull-secret-syncer-q8q7d\" (UID: \"5a6ad56c-e987-47b6-a64c-acf6705a3228\") " pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:56.613466 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.613400 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/672720fa-209d-4354-8698-27024f8e88ec-metrics-certs\") pod \"network-metrics-daemon-pfkrn\" (UID: \"672720fa-209d-4354-8698-27024f8e88ec\") " pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:56.617286 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.617271 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q8q7d" Jul 06 11:29:56.625262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.625248 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w64mh\"" Jul 06 11:29:56.633011 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.632986 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfkrn" Jul 06 11:29:56.746250 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.746222 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q8q7d"] Jul 06 11:29:56.751032 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:56.749522 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6ad56c_e987_47b6_a64c_acf6705a3228.slice/crio-7fe81eedb234ec35e725a61a48628a73d14d562b0b99c959e53126e0b686b532 WatchSource:0}: Error finding container 7fe81eedb234ec35e725a61a48628a73d14d562b0b99c959e53126e0b686b532: Status 404 returned error can't find the container with id 7fe81eedb234ec35e725a61a48628a73d14d562b0b99c959e53126e0b686b532 Jul 06 11:29:56.764875 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.764854 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pfkrn"] Jul 06 11:29:56.769290 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:56.769270 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672720fa_209d_4354_8698_27024f8e88ec.slice/crio-ffaa8c2384a5e77d9acc710a1cc9c0e55edaf91a37db77d54dc9a4b365bc04eb WatchSource:0}: Error finding container ffaa8c2384a5e77d9acc710a1cc9c0e55edaf91a37db77d54dc9a4b365bc04eb: Status 404 returned error can't find the container with id ffaa8c2384a5e77d9acc710a1cc9c0e55edaf91a37db77d54dc9a4b365bc04eb Jul 06 11:29:56.802223 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.802195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:56.804405 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.804387 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2skvc\" (UniqueName: \"kubernetes.io/projected/1f8a3be6-208b-46d8-a37e-4e058d9d0613-kube-api-access-2skvc\") pod \"network-check-target-66m6t\" (UID: \"1f8a3be6-208b-46d8-a37e-4e058d9d0613\") " pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:56.845023 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.845001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:56.912481 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.912431 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kdds4\"" Jul 06 11:29:56.914806 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.914788 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:56.920881 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:56.920868 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:57.045825 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.045797 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-66m6t"] Jul 06 11:29:57.048751 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:57.048728 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8a3be6_208b_46d8_a37e_4e058d9d0613.slice/crio-736d9fa0652b94644de35c156d88eefd0c27ffd4e80004e86b489d4e4f2f0b0c WatchSource:0}: Error finding container 736d9fa0652b94644de35c156d88eefd0c27ffd4e80004e86b489d4e4f2f0b0c: Status 404 returned error can't find the container with id 736d9fa0652b94644de35c156d88eefd0c27ffd4e80004e86b489d4e4f2f0b0c Jul 06 11:29:57.276750 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.276717 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-66m6t" event={"ID":"1f8a3be6-208b-46d8-a37e-4e058d9d0613","Type":"ContainerStarted","Data":"09d09fd6c2951454ec74a09be538fc332bb4103aeccf85e141e45cb1f8bcc683"} Jul 06 11:29:57.276873 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.276778 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-66m6t" event={"ID":"1f8a3be6-208b-46d8-a37e-4e058d9d0613","Type":"ContainerStarted","Data":"736d9fa0652b94644de35c156d88eefd0c27ffd4e80004e86b489d4e4f2f0b0c"} Jul 06 11:29:57.276873 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.276806 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:29:57.278140 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.278115 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q8q7d" event={"ID":"5a6ad56c-e987-47b6-a64c-acf6705a3228","Type":"ContainerStarted","Data":"7fe81eedb234ec35e725a61a48628a73d14d562b0b99c959e53126e0b686b532"} Jul 06 11:29:57.279351 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.279326 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfkrn" event={"ID":"672720fa-209d-4354-8698-27024f8e88ec","Type":"ContainerStarted","Data":"ffaa8c2384a5e77d9acc710a1cc9c0e55edaf91a37db77d54dc9a4b365bc04eb"} Jul 06 11:29:57.294245 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.294204 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-66m6t" podStartSLOduration=66.294188735 podStartE2EDuration="1m6.294188735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:57.293850376 +0000 UTC m=+66.825589702" watchObservedRunningTime="2026-07-06 11:29:57.294188735 +0000 UTC m=+66.825928055" Jul 06 11:29:57.608609 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.608530 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:57.611037 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.611009 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8052bc97-28d9-466e-87fa-5435fbd1baa1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xwdrh\" (UID: \"8052bc97-28d9-466e-87fa-5435fbd1baa1\") " pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:57.813005 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.812972 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hcchr\"" Jul 06 11:29:57.820951 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.820923 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xwdrh" Jul 06 11:29:57.995580 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:57.995552 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xwdrh"] Jul 06 11:29:58.000636 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:58.000608 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8052bc97_28d9_466e_87fa_5435fbd1baa1.slice/crio-61acd080cddce26a55bc8ed2cd5217999b55d31af0a180e50974a49b2938799d WatchSource:0}: Error finding container 61acd080cddce26a55bc8ed2cd5217999b55d31af0a180e50974a49b2938799d: Status 404 returned error can't find the container with id 61acd080cddce26a55bc8ed2cd5217999b55d31af0a180e50974a49b2938799d Jul 06 11:29:58.285982 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.285933 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfkrn" event={"ID":"672720fa-209d-4354-8698-27024f8e88ec","Type":"ContainerStarted","Data":"0e0478af43ea8a581877a8e65e713d9c1f5fedcc3f99c3276843e2b7e9e48ca6"} Jul 06 11:29:58.286150 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.285991 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfkrn" event={"ID":"672720fa-209d-4354-8698-27024f8e88ec","Type":"ContainerStarted","Data":"418e6ec7faf5d0b8c15d543b4581e2b7c00a55a9d376e8333e77ae28dca7e293"} Jul 06 11:29:58.287298 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.287268 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwdrh" event={"ID":"8052bc97-28d9-466e-87fa-5435fbd1baa1","Type":"ContainerStarted","Data":"61acd080cddce26a55bc8ed2cd5217999b55d31af0a180e50974a49b2938799d"} Jul 06 11:29:58.312449 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.312382 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pfkrn" podStartSLOduration=66.179987004 podStartE2EDuration="1m7.312366076s" totalImagesPullingTime="1.132379072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:29:58.311651954 +0000 UTC m=+67.843391308" watchObservedRunningTime="2026-07-06 11:29:58.312366076 +0000 UTC m=+67.844105402" Jul 06 11:29:58.654304 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.653553 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5"] Jul 06 11:29:58.657187 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.656941 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67b79c6d77-s52vt"] Jul 06 11:29:58.657187 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.657065 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.659968 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.659593 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 11:29:58.659968 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.659817 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 11:29:58.660185 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.660008 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-h4hmx\"" Jul 06 11:29:58.660315 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.660294 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8"] Jul 06 11:29:58.660413 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.660394 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.663833 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.663442 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.666025 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.665972 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 11:29:58.666127 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.666086 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 11:29:58.666232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.666213 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 11:29:58.666298 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.666255 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 11:29:58.669587 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.668401 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2fsgh\"" Jul 06 11:29:58.669587 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.668717 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 11:29:58.670152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.669833 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 11:29:58.670152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.670035 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rbs4n\"" Jul 06 11:29:58.671074 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.670985 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5"] Jul 06 11:29:58.671671 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.671215 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 11:29:58.673213 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.673029 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 11:29:58.682223 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.682178 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67b79c6d77-s52vt"] Jul 06 11:29:58.698147 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.698127 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8"] Jul 06 11:29:58.716576 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716297 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4tdp5\" (UID: \"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.716576 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716332 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4tdp5\" (UID: \"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.716576 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-ca-trust-extracted\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-image-registry-private-configuration\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716914 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-tls\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716939 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-bound-sa-token\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.716968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkcbl\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-kube-api-access-nkcbl\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.717001 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jknsd\" (UniqueName: \"kubernetes.io/projected/ed7d7df4-cad1-46be-8916-521c719d6c9a-kube-api-access-jknsd\") pod \"managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8\" (UID: \"ed7d7df4-cad1-46be-8916-521c719d6c9a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.717030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed7d7df4-cad1-46be-8916-521c719d6c9a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8\" (UID: \"ed7d7df4-cad1-46be-8916-521c719d6c9a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.717068 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-trusted-ca\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.717091 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-installation-pull-secrets\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.717287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.717151 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-certificates\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.751469 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.751215 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b79c6d77-s52vt"] Jul 06 11:29:58.751469 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:29:58.751425 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-nkcbl registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" podUID="e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" Jul 06 11:29:58.754819 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.754701 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-8bz8c"] Jul 06 11:29:58.758089 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.758069 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:29:58.760874 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.760855 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 11:29:58.761256 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.761065 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 11:29:58.761256 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.761128 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7wqkr\"" Jul 06 11:29:58.775007 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.774962 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-8bz8c"] Jul 06 11:29:58.815615 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.815593 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:58.818024 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.817996 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-tls\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818142 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818044 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-bound-sa-token\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818142 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818078 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz9pq\" (UniqueName: \"kubernetes.io/projected/c6bec83a-0303-41cd-b5f1-0677571af709-kube-api-access-nz9pq\") pod \"downloads-685bfb5f64-8bz8c\" (UID: \"c6bec83a-0303-41cd-b5f1-0677571af709\") " pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:29:58.818142 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818110 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkcbl\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-kube-api-access-nkcbl\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818147 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jknsd\" (UniqueName: \"kubernetes.io/projected/ed7d7df4-cad1-46be-8916-521c719d6c9a-kube-api-access-jknsd\") pod \"managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8\" (UID: \"ed7d7df4-cad1-46be-8916-521c719d6c9a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.818333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed7d7df4-cad1-46be-8916-521c719d6c9a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8\" (UID: \"ed7d7df4-cad1-46be-8916-521c719d6c9a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.818333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818228 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-trusted-ca\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818256 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-installation-pull-secrets\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818316 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-certificates\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4tdp5\" (UID: \"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.818630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818415 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4tdp5\" (UID: \"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.818630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818441 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-ca-trust-extracted\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.818630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.818470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-image-registry-private-configuration\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.820579 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.820255 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-4tdp5\" (UID: \"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.820579 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.820420 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-certificates\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.821436 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.821256 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-trusted-ca\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.821436 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.821262 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-ca-trust-extracted\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.825545 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.825505 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-4tdp5\" (UID: \"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:58.826124 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.826096 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-tls\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.826649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.826612 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-installation-pull-secrets\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.831040 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.830998 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed7d7df4-cad1-46be-8916-521c719d6c9a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8\" (UID: \"ed7d7df4-cad1-46be-8916-521c719d6c9a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.831804 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.831780 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-image-registry-private-configuration\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.841341 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.841251 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-bound-sa-token\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.841584 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.841529 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkcbl\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-kube-api-access-nkcbl\") pod \"image-registry-67b79c6d77-s52vt\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:58.842884 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.842846 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jknsd\" (UniqueName: \"kubernetes.io/projected/ed7d7df4-cad1-46be-8916-521c719d6c9a-kube-api-access-jknsd\") pod \"managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8\" (UID: \"ed7d7df4-cad1-46be-8916-521c719d6c9a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:58.912659 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.912514 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:58.919538 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.919199 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz9pq\" (UniqueName: \"kubernetes.io/projected/c6bec83a-0303-41cd-b5f1-0677571af709-kube-api-access-nz9pq\") pod \"downloads-685bfb5f64-8bz8c\" (UID: \"c6bec83a-0303-41cd-b5f1-0677571af709\") " pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:29:58.942347 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.942323 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz9pq\" (UniqueName: \"kubernetes.io/projected/c6bec83a-0303-41cd-b5f1-0677571af709-kube-api-access-nz9pq\") pod \"downloads-685bfb5f64-8bz8c\" (UID: \"c6bec83a-0303-41cd-b5f1-0677571af709\") " pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:29:58.972843 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:58.972822 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" Jul 06 11:29:59.001279 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.001251 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" Jul 06 11:29:59.009681 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.009616 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:59.070773 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.070742 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:29:59.112191 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.112110 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:59.170633 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.170512 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5"] Jul 06 11:29:59.182599 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:59.182569 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48682a5_19b3_4f9b_ac7f_0e4ec2f89bda.slice/crio-fac3af2aca68bd346435eb31cd3e2a5b486e2e6b9471392fce2d428097bca378 WatchSource:0}: Error finding container fac3af2aca68bd346435eb31cd3e2a5b486e2e6b9471392fce2d428097bca378: Status 404 returned error can't find the container with id fac3af2aca68bd346435eb31cd3e2a5b486e2e6b9471392fce2d428097bca378 Jul 06 11:29:59.206311 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.202132 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8"] Jul 06 11:29:59.206311 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.203934 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:59.270267 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.270237 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-8bz8c"] Jul 06 11:29:59.274877 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:29:59.274774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6bec83a_0303_41cd_b5f1_0677571af709.slice/crio-0f213f5e58fac8a1ac228f0a221c7a4e0d64662ee74bc555cf8b1b75551465ed WatchSource:0}: Error finding container 0f213f5e58fac8a1ac228f0a221c7a4e0d64662ee74bc555cf8b1b75551465ed: Status 404 returned error can't find the container with id 0f213f5e58fac8a1ac228f0a221c7a4e0d64662ee74bc555cf8b1b75551465ed Jul 06 11:29:59.292812 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.292782 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" event={"ID":"ed7d7df4-cad1-46be-8916-521c719d6c9a","Type":"ContainerStarted","Data":"d4fba49d9a717a72a049fc742c0909a94a588962ce8215befe0c663dd1b4ef4b"} Jul 06 11:29:59.294842 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.294804 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-8bz8c" event={"ID":"c6bec83a-0303-41cd-b5f1-0677571af709","Type":"ContainerStarted","Data":"0f213f5e58fac8a1ac228f0a221c7a4e0d64662ee74bc555cf8b1b75551465ed"} Jul 06 11:29:59.297454 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.297323 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" event={"ID":"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda","Type":"ContainerStarted","Data":"fac3af2aca68bd346435eb31cd3e2a5b486e2e6b9471392fce2d428097bca378"} Jul 06 11:29:59.297454 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.297406 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:59.304957 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.304939 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:29:59.427855 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.427787 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-ca-trust-extracted\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.427855 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.427835 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-tls\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428076 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.427856 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-certificates\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428076 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.427908 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkcbl\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-kube-api-access-nkcbl\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428076 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.427982 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-bound-sa-token\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428264 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.428111 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-image-registry-private-configuration\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428264 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.428142 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:29:59.428264 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.428150 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-installation-pull-secrets\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428264 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.428229 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-trusted-ca\") pod \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\" (UID: \"e76ecf4d-8c54-4ade-97fb-10c72aac3b1f\") " Jul 06 11:29:59.428543 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.428526 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-ca-trust-extracted\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.428624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.428563 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:29:59.429240 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.429213 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:29:59.430889 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.430862 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:29:59.431791 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.431731 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-kube-api-access-nkcbl" (OuterVolumeSpecName: "kube-api-access-nkcbl") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "kube-api-access-nkcbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:29:59.432422 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.432392 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:29:59.432913 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.432800 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:29:59.432913 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.432889 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" (UID: "e76ecf4d-8c54-4ade-97fb-10c72aac3b1f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:29:59.529439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529410 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-image-registry-private-configuration\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.529439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529439 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-installation-pull-secrets\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.529624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529456 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-trusted-ca\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.529624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529471 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-tls\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.529624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529485 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-registry-certificates\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.529624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529498 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkcbl\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-kube-api-access-nkcbl\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.529624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.529511 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f-bound-sa-token\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:29:59.616018 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.615983 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:29:59.676304 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:29:59.676281 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:00.237197 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.237151 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64f559b798-5zt2v"] Jul 06 11:30:00.240174 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.240144 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.244933 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.244914 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 11:30:00.245429 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.245408 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2vsbz\"" Jul 06 11:30:00.245429 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.245421 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 11:30:00.245624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.245471 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 11:30:00.247825 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.247801 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 11:30:00.248641 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.248625 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 11:30:00.253225 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.253209 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 11:30:00.262205 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.262181 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f559b798-5zt2v"] Jul 06 11:30:00.301323 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.301302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67b79c6d77-s52vt" Jul 06 11:30:00.336642 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336617 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-serving-cert\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.336752 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336657 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-config\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.336752 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336690 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmt56\" (UniqueName: \"kubernetes.io/projected/e6afc083-db02-461b-bd3e-65d6db51c5fd-kube-api-access-kmt56\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.336833 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336770 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-trusted-ca-bundle\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.336833 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336796 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-service-ca\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.336833 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336823 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-oauth-serving-cert\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.336958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.336867 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-oauth-config\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.339952 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.339908 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67b79c6d77-s52vt"] Jul 06 11:30:00.341759 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.341738 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67b79c6d77-s52vt"] Jul 06 11:30:00.438023 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.437994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-service-ca\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438150 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438034 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-oauth-serving-cert\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438150 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-oauth-config\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438150 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438102 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-serving-cert\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438348 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438200 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-config\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438348 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmt56\" (UniqueName: \"kubernetes.io/projected/e6afc083-db02-461b-bd3e-65d6db51c5fd-kube-api-access-kmt56\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438348 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-trusted-ca-bundle\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438846 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438793 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-oauth-serving-cert\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438929 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438875 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-service-ca\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.438929 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.438907 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-config\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.439284 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.439241 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-trusted-ca-bundle\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.440746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.440723 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-oauth-config\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.440931 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.440910 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-serving-cert\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.445840 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.445810 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmt56\" (UniqueName: \"kubernetes.io/projected/e6afc083-db02-461b-bd3e-65d6db51c5fd-kube-api-access-kmt56\") pod \"console-64f559b798-5zt2v\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.550365 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.550336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:00.645781 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.645752 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:00.997970 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:00.997934 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e76ecf4d-8c54-4ade-97fb-10c72aac3b1f" path="/var/lib/kubelet/pods/e76ecf4d-8c54-4ade-97fb-10c72aac3b1f/volumes" Jul 06 11:30:02.144703 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:02.144662 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:02.494897 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:02.494819 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:02.568003 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:02.567919 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:04.564504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:04.564425 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f559b798-5zt2v"] Jul 06 11:30:05.124032 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:05.123992 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:05.225419 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:05.225366 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:05.690344 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:05.690264 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6afc083_db02_461b_bd3e_65d6db51c5fd.slice/crio-51503dc2a8b177da04602611700ecacf74e8c62fddb9e6a16d902cf41a0bca78 WatchSource:0}: Error finding container 51503dc2a8b177da04602611700ecacf74e8c62fddb9e6a16d902cf41a0bca78: Status 404 returned error can't find the container with id 51503dc2a8b177da04602611700ecacf74e8c62fddb9e6a16d902cf41a0bca78 Jul 06 11:30:06.325780 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:06.325739 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f559b798-5zt2v" event={"ID":"e6afc083-db02-461b-bd3e-65d6db51c5fd","Type":"ContainerStarted","Data":"51503dc2a8b177da04602611700ecacf74e8c62fddb9e6a16d902cf41a0bca78"} Jul 06 11:30:06.501746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:06.501714 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:06.600875 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:06.600792 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:07.327296 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.327016 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:07.331485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.331332 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q8q7d" event={"ID":"5a6ad56c-e987-47b6-a64c-acf6705a3228","Type":"ContainerStarted","Data":"a25f0ac23767033d339c94bdaebb95aecdfe664c08639b527baeec940c4eeb74"} Jul 06 11:30:07.333539 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.333510 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwdrh" event={"ID":"8052bc97-28d9-466e-87fa-5435fbd1baa1","Type":"ContainerStarted","Data":"f1f17383129e946361a996158cbba005a555d45d73a4cf68f65afefec67c1e1f"} Jul 06 11:30:07.335330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.335305 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" event={"ID":"ed7d7df4-cad1-46be-8916-521c719d6c9a","Type":"ContainerStarted","Data":"9c40a67f980c1fe7216d1b36907e7e9dfdd172eff8aa94de01b6666f52974ce6"} Jul 06 11:30:07.336939 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.336915 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" event={"ID":"a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda","Type":"ContainerStarted","Data":"9def9a2bbe0036bffa4400a2df9c53f268e4a87cf0b82d29ea6964246192e2a2"} Jul 06 11:30:07.348599 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.348556 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q8q7d" podStartSLOduration=67.412573084 podStartE2EDuration="1m16.348537355s" totalImagesPullingTime="8.935964271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:07.347608866 +0000 UTC m=+76.879348193" watchObservedRunningTime="2026-07-06 11:30:07.348537355 +0000 UTC m=+76.880276684" Jul 06 11:30:07.364334 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.364271 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-4tdp5" podStartSLOduration=2.167142054 podStartE2EDuration="9.364252872s" totalImagesPullingTime="7.197110818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:07.363394817 +0000 UTC m=+76.895134144" watchObservedRunningTime="2026-07-06 11:30:07.364252872 +0000 UTC m=+76.895992199" Jul 06 11:30:07.387820 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.387778 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c8f6ff6cd-8cjl8" podStartSLOduration=2.180384674 podStartE2EDuration="9.387766057s" totalImagesPullingTime="7.207381383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:07.387669455 +0000 UTC m=+76.919408782" watchObservedRunningTime="2026-07-06 11:30:07.387766057 +0000 UTC m=+76.919505384" Jul 06 11:30:07.497001 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.496969 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:07.618601 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.618532 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:07.720509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:07.720471 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:08.343703 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:08.343534 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwdrh" event={"ID":"8052bc97-28d9-466e-87fa-5435fbd1baa1","Type":"ContainerStarted","Data":"dd337f73bce2361b369945d2d83c4c8b4430e998856a0d32de4096c6d39a2824"} Jul 06 11:30:09.422665 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:09.422629 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:09.524461 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:09.524425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:10.041346 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.041310 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:10.355276 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.355148 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f559b798-5zt2v" event={"ID":"e6afc083-db02-461b-bd3e-65d6db51c5fd","Type":"ContainerStarted","Data":"fbf3861ec778938866b67f2d6484f0babc99ae75d0731101541846ef2d7f4b75"} Jul 06 11:30:10.377092 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.377037 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64f559b798-5zt2v" podStartSLOduration=6.565550827 podStartE2EDuration="10.377020842s" totalImagesPullingTime="3.811470015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:10.376443875 +0000 UTC m=+79.908183202" watchObservedRunningTime="2026-07-06 11:30:10.377020842 +0000 UTC m=+79.908760168" Jul 06 11:30:10.504302 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.504228 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:10.551250 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.551220 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:10.551387 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.551276 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:10.557926 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.557873 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:10.599271 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:10.599239 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:11.362751 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.362709 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:11.541794 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.541752 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:11.667364 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.667291 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:11.743801 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.743775 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:11.850984 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.850935 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:11.911216 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.911182 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6"] Jul 06 11:30:11.927517 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.927452 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bsdpk"] Jul 06 11:30:11.928652 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.928634 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:11.932694 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.932566 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6"] Jul 06 11:30:11.932694 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.932686 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:11.935774 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.935498 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.938527 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.938749 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.938979 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.939190 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.939504 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9hqv9\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.939700 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 11:30:11.940958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.939936 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 11:30:11.941920 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.941753 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-85mzg\"" Jul 06 11:30:11.941920 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.940272 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 11:30:11.942122 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.942073 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv"] Jul 06 11:30:11.946258 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.946222 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:11.948267 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.948243 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:11.948517 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.948499 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 11:30:11.948616 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.948579 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 11:30:11.948748 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.948731 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 11:30:11.949348 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.949207 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bctc6\"" Jul 06 11:30:11.956122 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:11.956105 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv"] Jul 06 11:30:12.032304 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032269 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.032488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032316 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-tls\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032344 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.032488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032372 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf2rz\" (UniqueName: \"kubernetes.io/projected/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-kube-api-access-zf2rz\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032402 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-accelerators-collector-config\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032443 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.032488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032470 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-textfile\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032517 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032542 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g4pg\" (UniqueName: \"kubernetes.io/projected/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-api-access-9g4pg\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032564 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-wtmp\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032588 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-metrics-client-ca\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032615 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032643 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvvs9\" (UniqueName: \"kubernetes.io/projected/a3e9e6f5-6df7-4420-8180-585ed2f64161-kube-api-access-mvvs9\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032698 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-sys\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032729 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-root\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e9e6f5-6df7-4420-8180-585ed2f64161-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.032797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032795 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.033297 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032874 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.033297 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.032912 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.043624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.043599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:12.133250 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-textfile\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133392 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133255 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.133392 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g4pg\" (UniqueName: \"kubernetes.io/projected/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-api-access-9g4pg\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.133392 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-wtmp\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133392 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-metrics-client-ca\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133392 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133356 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.133392 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvvs9\" (UniqueName: \"kubernetes.io/projected/a3e9e6f5-6df7-4420-8180-585ed2f64161-kube-api-access-mvvs9\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-sys\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-root\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133472 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e9e6f5-6df7-4420-8180-585ed2f64161-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133503 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133566 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133592 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133598 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-textfile\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-tls\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133648 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133674 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-sys\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.133693 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf2rz\" (UniqueName: \"kubernetes.io/projected/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-kube-api-access-zf2rz\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.134280 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-accelerators-collector-config\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.134280 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.133767 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.134280 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:30:12.133876 2585 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 11:30:12.134280 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:30:12.133945 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-tls podName:5d3a800a-9d8f-4e0a-b917-d9186c24a12c nodeName:}" failed. No retries permitted until 2026-07-06 11:30:12.633926002 +0000 UTC m=+82.165665306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-hlxwv" (UID: "5d3a800a-9d8f-4e0a-b917-d9186c24a12c") : secret "kube-state-metrics-tls" not found Jul 06 11:30:12.134280 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.134012 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-root\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.134826 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.134727 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3e9e6f5-6df7-4420-8180-585ed2f64161-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.134826 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.134735 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-wtmp\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.135328 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:30:12.135187 2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 11:30:12.135328 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.135156 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-accelerators-collector-config\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.135328 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:30:12.135259 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-tls podName:a3e9e6f5-6df7-4420-8180-585ed2f64161 nodeName:}" failed. No retries permitted until 2026-07-06 11:30:12.635238471 +0000 UTC m=+82.166977795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-784f6" (UID: "a3e9e6f5-6df7-4420-8180-585ed2f64161") : secret "openshift-state-metrics-tls" not found Jul 06 11:30:12.135715 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.135690 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.135871 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.135841 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.135944 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.135887 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.136437 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.136411 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-metrics-client-ca\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.138083 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.138044 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-tls\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.139486 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.139445 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.139786 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.139761 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.140235 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.140191 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.142544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.142499 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:12.144540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.144516 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g4pg\" (UniqueName: \"kubernetes.io/projected/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-api-access-9g4pg\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.145092 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.145058 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvvs9\" (UniqueName: \"kubernetes.io/projected/a3e9e6f5-6df7-4420-8180-585ed2f64161-kube-api-access-mvvs9\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.145551 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.145526 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf2rz\" (UniqueName: \"kubernetes.io/projected/dcd49f24-f4df-48fc-848a-0fb607ee0f9b-kube-api-access-zf2rz\") pod \"node-exporter-bsdpk\" (UID: \"dcd49f24-f4df-48fc-848a-0fb607ee0f9b\") " pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.257012 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.256988 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bsdpk" Jul 06 11:30:12.307394 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.307345 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:12.403176 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.403142 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:12.505414 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.505385 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:12.637965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.637876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.637965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.637946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.641065 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.641039 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d3a800a-9d8f-4e0a-b917-d9186c24a12c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-hlxwv\" (UID: \"5d3a800a-9d8f-4e0a-b917-d9186c24a12c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.641201 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.641042 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3e9e6f5-6df7-4420-8180-585ed2f64161-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-784f6\" (UID: \"a3e9e6f5-6df7-4420-8180-585ed2f64161\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.846493 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.846459 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" Jul 06 11:30:12.863652 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.863630 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" Jul 06 11:30:12.893274 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.893191 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:12.992466 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:12.992352 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:13.020439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.020412 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:30:13.025195 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.025176 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.027671 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.027653 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 11:30:13.027775 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.027693 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 11:30:13.028010 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.027977 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 11:30:13.028107 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028042 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 11:30:13.028187 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028126 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 11:30:13.028187 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028143 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 11:30:13.028415 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028397 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 11:30:13.028495 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028413 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mt692\"" Jul 06 11:30:13.028495 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028413 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 11:30:13.028606 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.028526 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 11:30:13.043042 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.043021 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:30:13.083587 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.083553 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:13.142292 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142269 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142387 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142304 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142387 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142328 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-config-volume\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142387 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142518 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142518 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tln4q\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-kube-api-access-tln4q\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142518 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142566 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-config-out\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142614 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142700 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142700 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142666 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142787 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142703 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-web-config\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.142787 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.142732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.243412 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.243412 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243375 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tln4q\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-kube-api-access-tln4q\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.243795 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243768 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.243894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243836 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-config-out\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.243965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243881 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.243965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244086 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243949 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:13.244086 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.243966 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244086 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.244008 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-web-config\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244086 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.244058 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.244119 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.244145 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.244210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-config-volume\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.244331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.244234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.245625 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.245001 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.245625 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.245040 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.245625 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:30:13.245095 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle podName:ecff39fc-9b47-4622-aa57-4aa232f76764 nodeName:}" failed. No retries permitted until 2026-07-06 11:30:13.745072327 +0000 UTC m=+83.276811632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764") : configmap references non-existent config key: ca-bundle.crt Jul 06 11:30:13.247151 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.247106 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.247870 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.247696 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.247870 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.247820 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-web-config\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.248225 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.248184 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-config-out\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.248466 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.248443 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-config-volume\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.248541 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.248445 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.249219 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.249145 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.249387 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.249365 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.250350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.250325 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.252213 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.252192 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tln4q\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-kube-api-access-tln4q\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.749470 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.749435 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.750433 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.750402 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:13.937462 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:13.937432 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:30:14.935725 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:14.935686 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:15.024428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.024378 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:15.025408 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.025381 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-f5c478986-z444g"] Jul 06 11:30:15.030649 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.030623 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.033095 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.032879 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 11:30:15.033095 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.032879 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 11:30:15.033095 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.032985 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 11:30:15.033415 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.033381 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2vj7tsjt6qb55\"" Jul 06 11:30:15.033600 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.033571 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 11:30:15.033691 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.033666 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tkdv5\"" Jul 06 11:30:15.033753 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.033576 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 11:30:15.042137 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.042118 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f5c478986-z444g"] Jul 06 11:30:15.159750 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.159725 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.159750 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.159763 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvbcm\" (UniqueName: \"kubernetes.io/projected/edfbdc71-00c7-40e7-840a-4958e4c55c5d-kube-api-access-xvbcm\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.159982 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.159794 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.159982 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.159835 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.159982 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.159910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-grpc-tls\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.160129 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.160029 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-tls\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.160129 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.160063 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.160129 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.160091 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edfbdc71-00c7-40e7-840a-4958e4c55c5d-metrics-client-ca\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.161097 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.161077 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:15.261066 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261081 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvbcm\" (UniqueName: \"kubernetes.io/projected/edfbdc71-00c7-40e7-840a-4958e4c55c5d-kube-api-access-xvbcm\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261107 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261148 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261213 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-grpc-tls\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261451 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-tls\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261451 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261297 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.261451 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.261319 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edfbdc71-00c7-40e7-840a-4958e4c55c5d-metrics-client-ca\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.262483 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.262065 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edfbdc71-00c7-40e7-840a-4958e4c55c5d-metrics-client-ca\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.264288 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.264246 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.264521 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.264387 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-tls\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.264521 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.264460 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-grpc-tls\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.265309 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.265284 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.265423 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.265345 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.265702 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.265681 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/edfbdc71-00c7-40e7-840a-4958e4c55c5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.270612 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.270590 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvbcm\" (UniqueName: \"kubernetes.io/projected/edfbdc71-00c7-40e7-840a-4958e4c55c5d-kube-api-access-xvbcm\") pod \"thanos-querier-f5c478986-z444g\" (UID: \"edfbdc71-00c7-40e7-840a-4958e4c55c5d\") " pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.343487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.343457 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:15.390241 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:15.390215 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:16.462427 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.462400 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-57df598f5-v2n47"] Jul 06 11:30:16.466582 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.466560 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.468953 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.468931 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 11:30:16.469442 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.469378 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-43euklt7ihpfm\"" Jul 06 11:30:16.469556 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.469458 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 11:30:16.469556 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.469464 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 11:30:16.469556 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.469508 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 11:30:16.469556 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.469538 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kzhfd\"" Jul 06 11:30:16.475744 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.475727 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-57df598f5-v2n47"] Jul 06 11:30:16.571543 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5432e6ef-38c8-48e6-b092-8562c8ae94a2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.571689 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4z7\" (UniqueName: \"kubernetes.io/projected/5432e6ef-38c8-48e6-b092-8562c8ae94a2-kube-api-access-7v4z7\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.571689 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571589 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-secret-metrics-server-tls\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.571689 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5432e6ef-38c8-48e6-b092-8562c8ae94a2-metrics-server-audit-profiles\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.571852 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571722 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5432e6ef-38c8-48e6-b092-8562c8ae94a2-audit-log\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.571852 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571786 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-client-ca-bundle\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.571927 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.571861 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-secret-metrics-server-client-certs\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.672899 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.672870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5432e6ef-38c8-48e6-b092-8562c8ae94a2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.672933 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4z7\" (UniqueName: \"kubernetes.io/projected/5432e6ef-38c8-48e6-b092-8562c8ae94a2-kube-api-access-7v4z7\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.672975 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-secret-metrics-server-tls\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.673023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5432e6ef-38c8-48e6-b092-8562c8ae94a2-metrics-server-audit-profiles\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673269 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.673249 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5432e6ef-38c8-48e6-b092-8562c8ae94a2-audit-log\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673342 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.673286 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-client-ca-bundle\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673406 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.673348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-secret-metrics-server-client-certs\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673928 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.673690 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5432e6ef-38c8-48e6-b092-8562c8ae94a2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.673928 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.673886 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5432e6ef-38c8-48e6-b092-8562c8ae94a2-audit-log\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.674215 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.674191 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5432e6ef-38c8-48e6-b092-8562c8ae94a2-metrics-server-audit-profiles\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.676294 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.676256 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-secret-metrics-server-client-certs\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.677129 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.677108 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-client-ca-bundle\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.678113 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.678067 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5432e6ef-38c8-48e6-b092-8562c8ae94a2-secret-metrics-server-tls\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.683697 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.683675 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4z7\" (UniqueName: \"kubernetes.io/projected/5432e6ef-38c8-48e6-b092-8562c8ae94a2-kube-api-access-7v4z7\") pod \"metrics-server-57df598f5-v2n47\" (UID: \"5432e6ef-38c8-48e6-b092-8562c8ae94a2\") " pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.715683 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.715627 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:16.723509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.723490 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp"] Jul 06 11:30:16.728242 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.728223 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:16.730505 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.730270 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 11:30:16.730505 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.730291 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bkv8v\"" Jul 06 11:30:16.740986 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.740964 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp"] Jul 06 11:30:16.779006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.778979 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:16.806428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.806398 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:16.874831 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.874805 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2cnq\" (UniqueName: \"kubernetes.io/projected/c9eae277-ad42-4b0c-862d-0023de33d021-kube-api-access-m2cnq\") pod \"monitoring-plugin-d5dcff7cf-plzsp\" (UID: \"c9eae277-ad42-4b0c-862d-0023de33d021\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:16.874943 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.874851 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9eae277-ad42-4b0c-862d-0023de33d021-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-plzsp\" (UID: \"c9eae277-ad42-4b0c-862d-0023de33d021\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:16.907116 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.907090 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:16.975303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.975225 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2cnq\" (UniqueName: \"kubernetes.io/projected/c9eae277-ad42-4b0c-862d-0023de33d021-kube-api-access-m2cnq\") pod \"monitoring-plugin-d5dcff7cf-plzsp\" (UID: \"c9eae277-ad42-4b0c-862d-0023de33d021\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:16.975303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.975265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9eae277-ad42-4b0c-862d-0023de33d021-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-plzsp\" (UID: \"c9eae277-ad42-4b0c-862d-0023de33d021\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:16.977943 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.977921 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c9eae277-ad42-4b0c-862d-0023de33d021-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-plzsp\" (UID: \"c9eae277-ad42-4b0c-862d-0023de33d021\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:16.983777 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:16.983732 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2cnq\" (UniqueName: \"kubernetes.io/projected/c9eae277-ad42-4b0c-862d-0023de33d021-kube-api-access-m2cnq\") pod \"monitoring-plugin-d5dcff7cf-plzsp\" (UID: \"c9eae277-ad42-4b0c-862d-0023de33d021\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:17.023329 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.023281 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:17.040710 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.040684 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:17.118080 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.118057 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:17.332639 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.332601 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-86c9b55c95-ndw48"] Jul 06 11:30:17.339440 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.339420 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.348562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.348544 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 11:30:17.348711 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.348687 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 11:30:17.348959 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.348939 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 11:30:17.349629 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.349611 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 11:30:17.350064 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.350044 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c5lhf\"" Jul 06 11:30:17.354990 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.354945 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 11:30:17.364577 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.364559 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 11:30:17.380166 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.380139 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86c9b55c95-ndw48"] Jul 06 11:30:17.479662 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479637 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-secret-telemeter-client\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479701 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-federate-client-tls\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479770 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-serving-certs-ca-bundle\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479807 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479852 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-telemeter-client-tls\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479878 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-metrics-client-ca\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqh7p\" (UniqueName: \"kubernetes.io/projected/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-kube-api-access-sqh7p\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.480039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.479937 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.580458 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580428 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqh7p\" (UniqueName: \"kubernetes.io/projected/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-kube-api-access-sqh7p\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.580604 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.580604 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580534 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-secret-telemeter-client\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.580604 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580597 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-federate-client-tls\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.580757 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580656 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-serving-certs-ca-bundle\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.581345 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580903 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.581345 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580948 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-telemeter-client-tls\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.581345 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.580980 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-metrics-client-ca\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.581579 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.581402 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-serving-certs-ca-bundle\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.581579 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.581452 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.581689 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.581654 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-metrics-client-ca\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.583653 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.583551 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-secret-telemeter-client\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.583653 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.583581 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-federate-client-tls\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.583904 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.583883 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.584105 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.584082 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-telemeter-client-tls\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.594475 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.594451 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqh7p\" (UniqueName: \"kubernetes.io/projected/0a0619f4-a51c-4c49-a6b3-d5e297a2f58b-kube-api-access-sqh7p\") pod \"telemeter-client-86c9b55c95-ndw48\" (UID: \"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b\") " pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:17.650303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:17.650281 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" Jul 06 11:30:18.228000 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:18.227956 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:19.746540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:19.746008 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6"] Jul 06 11:30:19.754677 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:19.754653 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e9e6f5_6df7_4420_8180_585ed2f64161.slice/crio-b319f3381e25e1a815ad2aed033a5fccc51ffa35feab4b94fd663ce1c8304c5b WatchSource:0}: Error finding container b319f3381e25e1a815ad2aed033a5fccc51ffa35feab4b94fd663ce1c8304c5b: Status 404 returned error can't find the container with id b319f3381e25e1a815ad2aed033a5fccc51ffa35feab4b94fd663ce1c8304c5b Jul 06 11:30:19.789178 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:19.788473 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-57df598f5-v2n47"] Jul 06 11:30:19.991986 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:19.991954 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv"] Jul 06 11:30:19.998752 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:19.998730 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3a800a_9d8f_4e0a_b917_d9186c24a12c.slice/crio-55b63e4163957c31de33b54b7ec2084790e582b80d991cf785c173eba555d246 WatchSource:0}: Error finding container 55b63e4163957c31de33b54b7ec2084790e582b80d991cf785c173eba555d246: Status 404 returned error can't find the container with id 55b63e4163957c31de33b54b7ec2084790e582b80d991cf785c173eba555d246 Jul 06 11:30:20.033386 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.033298 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp"] Jul 06 11:30:20.036069 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:20.036047 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9eae277_ad42_4b0c_862d_0023de33d021.slice/crio-53cdbeff767520f891fe8e7b90581a309cba4fdb9d558da6e1f6bcf7e46f0491 WatchSource:0}: Error finding container 53cdbeff767520f891fe8e7b90581a309cba4fdb9d558da6e1f6bcf7e46f0491: Status 404 returned error can't find the container with id 53cdbeff767520f891fe8e7b90581a309cba4fdb9d558da6e1f6bcf7e46f0491 Jul 06 11:30:20.037999 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.037883 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f5c478986-z444g"] Jul 06 11:30:20.040211 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.040189 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86c9b55c95-ndw48"] Jul 06 11:30:20.040663 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:20.040628 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedfbdc71_00c7_40e7_840a_4958e4c55c5d.slice/crio-8a4b29a67aed6261f1ff9445025ce963e02f8468807b04440d69836deff46e00 WatchSource:0}: Error finding container 8a4b29a67aed6261f1ff9445025ce963e02f8468807b04440d69836deff46e00: Status 404 returned error can't find the container with id 8a4b29a67aed6261f1ff9445025ce963e02f8468807b04440d69836deff46e00 Jul 06 11:30:20.042944 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:20.042923 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0619f4_a51c_4c49_a6b3_d5e297a2f58b.slice/crio-a9b56649a0edf05282739b8402aa8904a3cec7a474723c4aaf92d7df175525d9 WatchSource:0}: Error finding container a9b56649a0edf05282739b8402aa8904a3cec7a474723c4aaf92d7df175525d9: Status 404 returned error can't find the container with id a9b56649a0edf05282739b8402aa8904a3cec7a474723c4aaf92d7df175525d9 Jul 06 11:30:20.045714 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.045689 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:30:20.049275 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:20.049255 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecff39fc_9b47_4622_aa57_4aa232f76764.slice/crio-9d4cad11d4d06364cff6cc7ebd0ae09deed6283f2e2f77a7bc439964109955e7 WatchSource:0}: Error finding container 9d4cad11d4d06364cff6cc7ebd0ae09deed6283f2e2f77a7bc439964109955e7: Status 404 returned error can't find the container with id 9d4cad11d4d06364cff6cc7ebd0ae09deed6283f2e2f77a7bc439964109955e7 Jul 06 11:30:20.231767 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.231711 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:20.332524 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.332491 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:20.394283 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.394199 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"9d4cad11d4d06364cff6cc7ebd0ae09deed6283f2e2f77a7bc439964109955e7"} Jul 06 11:30:20.396010 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.395911 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdpk" event={"ID":"dcd49f24-f4df-48fc-848a-0fb607ee0f9b","Type":"ContainerStarted","Data":"de0460905a4bf88ebdaa63df023008c2bfd81c2df7f45cbfb29b2949c96c5998"} Jul 06 11:30:20.399341 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.399286 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" event={"ID":"a3e9e6f5-6df7-4420-8180-585ed2f64161","Type":"ContainerStarted","Data":"fe2e72c5fb455f934ce059b0b9e29dd81b9e657332c2cd3427889fab8d7b7860"} Jul 06 11:30:20.399460 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.399355 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" event={"ID":"a3e9e6f5-6df7-4420-8180-585ed2f64161","Type":"ContainerStarted","Data":"2f825b5d9fe24a560a260a3d68b625e81942d86d5633fe9f7c921e3c4e435e25"} Jul 06 11:30:20.399460 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.399388 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" event={"ID":"a3e9e6f5-6df7-4420-8180-585ed2f64161","Type":"ContainerStarted","Data":"b319f3381e25e1a815ad2aed033a5fccc51ffa35feab4b94fd663ce1c8304c5b"} Jul 06 11:30:20.401425 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.401375 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" event={"ID":"c9eae277-ad42-4b0c-862d-0023de33d021","Type":"ContainerStarted","Data":"53cdbeff767520f891fe8e7b90581a309cba4fdb9d558da6e1f6bcf7e46f0491"} Jul 06 11:30:20.403260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.403230 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" event={"ID":"5d3a800a-9d8f-4e0a-b917-d9186c24a12c","Type":"ContainerStarted","Data":"55b63e4163957c31de33b54b7ec2084790e582b80d991cf785c173eba555d246"} Jul 06 11:30:20.407136 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.407084 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xwdrh" event={"ID":"8052bc97-28d9-466e-87fa-5435fbd1baa1","Type":"ContainerStarted","Data":"af9edff42f6ed68cc0ca95a576efd266767e1c59684d3239d8b20c2916a9bc23"} Jul 06 11:30:20.409355 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.409327 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"8a4b29a67aed6261f1ff9445025ce963e02f8468807b04440d69836deff46e00"} Jul 06 11:30:20.411024 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.410968 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" event={"ID":"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b","Type":"ContainerStarted","Data":"a9b56649a0edf05282739b8402aa8904a3cec7a474723c4aaf92d7df175525d9"} Jul 06 11:30:20.413494 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.413410 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-8bz8c" event={"ID":"c6bec83a-0303-41cd-b5f1-0677571af709","Type":"ContainerStarted","Data":"3c40e53179f63aae684be847725db9c066a5390352a02aacf3bdbb4c17a1e1c7"} Jul 06 11:30:20.413655 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.413634 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:30:20.414661 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.414563 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" event={"ID":"5432e6ef-38c8-48e6-b092-8562c8ae94a2","Type":"ContainerStarted","Data":"b5f42f6895cb38a2e386d4046c64da3abfb1a9d925a7d604649b45b9125b271d"} Jul 06 11:30:20.426068 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.426022 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xwdrh" podStartSLOduration=34.117634691 podStartE2EDuration="55.426005166s" totalImagesPullingTime="21.308370475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 11:29:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:20.424774409 +0000 UTC m=+89.956513764" watchObservedRunningTime="2026-07-06 11:30:20.426005166 +0000 UTC m=+89.957744492" Jul 06 11:30:20.427785 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.427749 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-8bz8c" Jul 06 11:30:20.430680 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.430645 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:20.442322 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:20.442282 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-8bz8c" podStartSLOduration=2.111279964 podStartE2EDuration="22.442264702s" totalImagesPullingTime="20.330984738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:20.440603863 +0000 UTC m=+89.972343190" watchObservedRunningTime="2026-07-06 11:30:20.442264702 +0000 UTC m=+89.974004029" Jul 06 11:30:21.352565 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:21.352416 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:21.462309 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:21.462229 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:23.443357 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:23.443315 2585 generic.go:356] "Generic (PLEG): container finished" podID="dcd49f24-f4df-48fc-848a-0fb607ee0f9b" containerID="d1a13c95c551068371ba46cce72ee571d894edf0d05a83758b62790a706d2ee6" exitCode=0 Jul 06 11:30:23.443773 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:23.443405 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdpk" event={"ID":"dcd49f24-f4df-48fc-848a-0fb607ee0f9b","Type":"ContainerDied","Data":"d1a13c95c551068371ba46cce72ee571d894edf0d05a83758b62790a706d2ee6"} Jul 06 11:30:24.489996 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:24.489957 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:24.590081 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:24.590052 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:27.143095 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.143028 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:27.245152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.245100 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:27.404892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.404763 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f559b798-5zt2v"] Jul 06 11:30:27.468426 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.466925 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e" exitCode=0 Jul 06 11:30:27.468426 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.467013 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e"} Jul 06 11:30:27.479184 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.474386 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdpk" event={"ID":"dcd49f24-f4df-48fc-848a-0fb607ee0f9b","Type":"ContainerStarted","Data":"60b508430c7da8f64a5a1eda14dcb82636d38345741480918e914cef33047ed1"} Jul 06 11:30:27.479184 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.474442 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bsdpk" event={"ID":"dcd49f24-f4df-48fc-848a-0fb607ee0f9b","Type":"ContainerStarted","Data":"8d31eec6522133e47ebf8adc647ea23512c2176528be0cc1f5fd7c7f3c3c5931"} Jul 06 11:30:27.482366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.481957 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" event={"ID":"a3e9e6f5-6df7-4420-8180-585ed2f64161","Type":"ContainerStarted","Data":"4cd7ac86d7d8b310964b1998eb77bb9365ead7a34813f3dec7d728fd21b1edfd"} Jul 06 11:30:27.485297 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.485241 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" event={"ID":"c9eae277-ad42-4b0c-862d-0023de33d021","Type":"ContainerStarted","Data":"71267df76ce3d0a4e569d9e89a45a5c7ab1d2ee4ee09f0c10b6bb85c96317fb0"} Jul 06 11:30:27.487806 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.486414 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:27.488487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.488460 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" event={"ID":"5d3a800a-9d8f-4e0a-b917-d9186c24a12c","Type":"ContainerStarted","Data":"d279b15cb76734754a7ac002e419e1a4d112cf400446a047d857bf282dcb8159"} Jul 06 11:30:27.497382 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.495998 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" Jul 06 11:30:27.511770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.511678 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"39e8c530ea7ab909c131228ea20f02803ffd74d3e08f707aff03b6fd37b8c612"} Jul 06 11:30:27.513675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.513604 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" event={"ID":"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b","Type":"ContainerStarted","Data":"af604c404678c797b3e75d184bb0826f9362179b023448de5bb6e854db59cae2"} Jul 06 11:30:27.517296 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.516676 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" event={"ID":"5432e6ef-38c8-48e6-b092-8562c8ae94a2","Type":"ContainerStarted","Data":"acb33b7890bdc9011090ab35dde6b1c2c87b279f2e8ef449e332fa2d64cf44b1"} Jul 06 11:30:27.543783 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.543729 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-784f6" podStartSLOduration=9.666711475 podStartE2EDuration="16.543702834s" totalImagesPullingTime="6.876991359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:27.542128223 +0000 UTC m=+97.073867543" watchObservedRunningTime="2026-07-06 11:30:27.543702834 +0000 UTC m=+97.075442163" Jul 06 11:30:27.548426 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.548387 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:27.598817 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.597638 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bsdpk" podStartSLOduration=14.712062686 podStartE2EDuration="16.597618693s" totalImagesPullingTime="1.794443749s" totalInitContainerRuntime="91.112258ms" isStatefulPod=true podCreationTimestamp="2026-07-06 11:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:27.57440787 +0000 UTC m=+97.106147197" watchObservedRunningTime="2026-07-06 11:30:27.597618693 +0000 UTC m=+97.129358020" Jul 06 11:30:27.602768 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.602727 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-plzsp" podStartSLOduration=4.72655499 podStartE2EDuration="11.602707606s" totalImagesPullingTime="6.876152616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:27.598637571 +0000 UTC m=+97.130377111" watchObservedRunningTime="2026-07-06 11:30:27.602707606 +0000 UTC m=+97.134446930" Jul 06 11:30:27.631565 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.631353 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" podStartSLOduration=4.5194304370000005 podStartE2EDuration="11.631334696s" totalImagesPullingTime="7.111904259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:27.629910579 +0000 UTC m=+97.161649907" watchObservedRunningTime="2026-07-06 11:30:27.631334696 +0000 UTC m=+97.163074023" Jul 06 11:30:27.648188 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.643350 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:27.711218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.711122 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:27.810872 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:27.810836 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:28.290848 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.290023 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-66m6t" Jul 06 11:30:28.527747 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.527709 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" event={"ID":"5d3a800a-9d8f-4e0a-b917-d9186c24a12c","Type":"ContainerStarted","Data":"2496ebb3f82fe92091af87734392f73c4f0a3560a835c98532a5ca6b59a395be"} Jul 06 11:30:28.527960 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.527781 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" event={"ID":"5d3a800a-9d8f-4e0a-b917-d9186c24a12c","Type":"ContainerStarted","Data":"f9a7eb0d2291787caee49911b5c3a27a0d975ade047dabad2b3f8cdf4f0daefd"} Jul 06 11:30:28.532094 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.532060 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"d3a8f0d257bc9cbac58144284a4a31fa34b795b8fd5d68e1455e1b692dab63b5"} Jul 06 11:30:28.532275 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.532136 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"f4175db7f701f9230a2baa8992dd7aff92dd75b56e9c716d6d771897313c9216"} Jul 06 11:30:28.535761 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.535724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" event={"ID":"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b","Type":"ContainerStarted","Data":"b12ef0dbfa24581d0e8addbe535027c4f454358f08a78616eeee5e2439102bcb"} Jul 06 11:30:28.535868 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.535785 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" event={"ID":"0a0619f4-a51c-4c49-a6b3-d5e297a2f58b","Type":"ContainerStarted","Data":"0d21d1fe8ebd50fce882cbf0876940ed84d8c59c46fa8ba5c76630cfe932455b"} Jul 06 11:30:28.548219 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.548122 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-hlxwv" podStartSLOduration=10.632893516 podStartE2EDuration="17.548103878s" totalImagesPullingTime="6.915210362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:28.547108699 +0000 UTC m=+98.078848025" watchObservedRunningTime="2026-07-06 11:30:28.548103878 +0000 UTC m=+98.079843198" Jul 06 11:30:28.574135 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:28.573586 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-86c9b55c95-ndw48" podStartSLOduration=4.704995208 podStartE2EDuration="11.573568942s" totalImagesPullingTime="6.868573734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:28.57180907 +0000 UTC m=+98.103548398" watchObservedRunningTime="2026-07-06 11:30:28.573568942 +0000 UTC m=+98.105308272" Jul 06 11:30:29.592467 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.592442 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58d9b4d489-rgdv2"] Jul 06 11:30:29.605205 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.605108 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.607452 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.607432 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d9b4d489-rgdv2"] Jul 06 11:30:29.709032 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709000 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lxw\" (UniqueName: \"kubernetes.io/projected/62a211a7-b162-49b5-acb8-a3ca2fc49868-kube-api-access-84lxw\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.709151 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709049 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-service-ca\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.709151 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709075 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-config\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.709151 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709120 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-trusted-ca-bundle\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.709328 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709180 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-oauth-serving-cert\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.709328 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709208 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-oauth-config\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.709328 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.709240 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-serving-cert\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.732174 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.732140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:29.810086 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-oauth-serving-cert\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810234 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810099 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-oauth-config\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810234 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810129 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-serving-cert\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810234 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lxw\" (UniqueName: \"kubernetes.io/projected/62a211a7-b162-49b5-acb8-a3ca2fc49868-kube-api-access-84lxw\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810246 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-service-ca\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-config\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810323 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-trusted-ca-bundle\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.810827 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810805 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-oauth-serving-cert\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.811002 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.810975 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-service-ca\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.811053 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.811013 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-trusted-ca-bundle\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.811140 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.811117 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-config\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.812975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.812955 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-oauth-config\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.813053 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.813042 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-serving-cert\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.818696 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.818672 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lxw\" (UniqueName: \"kubernetes.io/projected/62a211a7-b162-49b5-acb8-a3ca2fc49868-kube-api-access-84lxw\") pod \"console-58d9b4d489-rgdv2\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:29.831793 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.831771 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:29.915156 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:29.915135 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:30.087419 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.087282 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d9b4d489-rgdv2"] Jul 06 11:30:30.089549 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:30:30.089526 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a211a7_b162_49b5_acb8_a3ca2fc49868.slice/crio-9510ef7480c56754d07eb5115fc1f23378ba74ee768cd119d24a1ecf708f4483 WatchSource:0}: Error finding container 9510ef7480c56754d07eb5115fc1f23378ba74ee768cd119d24a1ecf708f4483: Status 404 returned error can't find the container with id 9510ef7480c56754d07eb5115fc1f23378ba74ee768cd119d24a1ecf708f4483 Jul 06 11:30:30.548685 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.548653 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"86780e0658481a9ab7b68fee7c7c0a80cb5c133a8ccc9f7a9aa8e078f8487f4a"} Jul 06 11:30:30.548806 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.548736 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"53b145a68c5560ac042bde9fb8b7cef096ae7036df4576ba8dcc85b567013bc6"} Jul 06 11:30:30.548806 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.548786 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" event={"ID":"edfbdc71-00c7-40e7-840a-4958e4c55c5d","Type":"ContainerStarted","Data":"e175356d1042192100707d4dc2f965cea54634e6e3a2f8e49e86653c72229f56"} Jul 06 11:30:30.548935 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.548835 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:30.552381 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.552330 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69"} Jul 06 11:30:30.552506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.552399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce"} Jul 06 11:30:30.552506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.552443 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd"} Jul 06 11:30:30.552506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.552484 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206"} Jul 06 11:30:30.552672 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.552521 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03"} Jul 06 11:30:30.554268 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.554243 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9b4d489-rgdv2" event={"ID":"62a211a7-b162-49b5-acb8-a3ca2fc49868","Type":"ContainerStarted","Data":"e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851"} Jul 06 11:30:30.554349 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.554276 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9b4d489-rgdv2" event={"ID":"62a211a7-b162-49b5-acb8-a3ca2fc49868","Type":"ContainerStarted","Data":"9510ef7480c56754d07eb5115fc1f23378ba74ee768cd119d24a1ecf708f4483"} Jul 06 11:30:30.576288 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.576203 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" podStartSLOduration=6.929031917 podStartE2EDuration="15.576184732s" totalImagesPullingTime="8.647152815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:30.573815694 +0000 UTC m=+100.105555036" watchObservedRunningTime="2026-07-06 11:30:30.576184732 +0000 UTC m=+100.107924060" Jul 06 11:30:30.595081 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.595037 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58d9b4d489-rgdv2" podStartSLOduration=1.595022749 podStartE2EDuration="1.595022749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:30.593676206 +0000 UTC m=+100.125415526" watchObservedRunningTime="2026-07-06 11:30:30.595022749 +0000 UTC m=+100.126762078" Jul 06 11:30:30.707340 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:30.707305 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:31.561770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:31.561739 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerStarted","Data":"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3"} Jul 06 11:30:31.597481 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:31.597429 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.064547421 podStartE2EDuration="18.597409309s" totalImagesPullingTime="9.50407603s" totalInitContainerRuntime="28.785858ms" isStatefulPod=false podCreationTimestamp="2026-07-06 11:30:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:30:31.596043098 +0000 UTC m=+101.127782425" watchObservedRunningTime="2026-07-06 11:30:31.597409309 +0000 UTC m=+101.129148636" Jul 06 11:30:34.709686 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:34.709654 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:34.812894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:34.812869 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:36.568495 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:36.568463 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-f5c478986-z444g" Jul 06 11:30:36.779930 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:36.779902 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:36.780123 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:36.779955 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:39.527631 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:39.527591 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:39.624627 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:39.624601 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:39.915352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:39.915283 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:39.915352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:39.915331 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:39.920276 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:39.920253 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:40.603281 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:40.603258 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:30:40.830572 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:40.830549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:40.936494 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:40.936429 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:41.009891 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:41.009857 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:52.479708 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.479635 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-64f559b798-5zt2v" podUID="e6afc083-db02-461b-bd3e-65d6db51c5fd" containerName="console" containerID="cri-o://fbf3861ec778938866b67f2d6484f0babc99ae75d0731101541846ef2d7f4b75" gracePeriod=15 Jul 06 11:30:52.648547 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.648512 2585 generic.go:356] "Generic (PLEG): container finished" podID="e6afc083-db02-461b-bd3e-65d6db51c5fd" containerID="fbf3861ec778938866b67f2d6484f0babc99ae75d0731101541846ef2d7f4b75" exitCode=0 Jul 06 11:30:52.648673 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.648583 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f559b798-5zt2v" event={"ID":"e6afc083-db02-461b-bd3e-65d6db51c5fd","Type":"ContainerDied","Data":"fbf3861ec778938866b67f2d6484f0babc99ae75d0731101541846ef2d7f4b75"} Jul 06 11:30:52.746051 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.746032 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:52.782200 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782182 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-service-ca\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782289 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782223 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-oauth-serving-cert\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782289 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782253 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-serving-cert\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782359 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782316 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmt56\" (UniqueName: \"kubernetes.io/projected/e6afc083-db02-461b-bd3e-65d6db51c5fd-kube-api-access-kmt56\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782395 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782355 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-oauth-config\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782395 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782384 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-config\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782464 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782409 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-trusted-ca-bundle\") pod \"e6afc083-db02-461b-bd3e-65d6db51c5fd\" (UID: \"e6afc083-db02-461b-bd3e-65d6db51c5fd\") " Jul 06 11:30:52.782890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782547 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-service-ca" (OuterVolumeSpecName: "service-ca") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:30:52.782890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782754 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-service-ca\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:52.782890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782757 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-config" (OuterVolumeSpecName: "console-config") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:30:52.782890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782845 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:30:52.782890 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.782869 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:30:52.784591 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.784563 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6afc083-db02-461b-bd3e-65d6db51c5fd-kube-api-access-kmt56" (OuterVolumeSpecName: "kube-api-access-kmt56") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "kube-api-access-kmt56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:30:52.784700 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.784647 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:30:52.784764 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.784733 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e6afc083-db02-461b-bd3e-65d6db51c5fd" (UID: "e6afc083-db02-461b-bd3e-65d6db51c5fd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:30:52.883705 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.883682 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-oauth-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:52.883705 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.883704 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:52.883867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.883714 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-trusted-ca-bundle\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:52.883867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.883723 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6afc083-db02-461b-bd3e-65d6db51c5fd-oauth-serving-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:52.883867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.883731 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6afc083-db02-461b-bd3e-65d6db51c5fd-console-serving-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:52.883867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:52.883739 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmt56\" (UniqueName: \"kubernetes.io/projected/e6afc083-db02-461b-bd3e-65d6db51c5fd-kube-api-access-kmt56\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:30:53.654268 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:53.654228 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f559b798-5zt2v" event={"ID":"e6afc083-db02-461b-bd3e-65d6db51c5fd","Type":"ContainerDied","Data":"51503dc2a8b177da04602611700ecacf74e8c62fddb9e6a16d902cf41a0bca78"} Jul 06 11:30:53.654268 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:53.654254 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f559b798-5zt2v" Jul 06 11:30:53.654706 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:53.654281 2585 scope.go:117] "RemoveContainer" containerID="fbf3861ec778938866b67f2d6484f0babc99ae75d0731101541846ef2d7f4b75" Jul 06 11:30:53.692582 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:53.692556 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f559b798-5zt2v"] Jul 06 11:30:53.696232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:53.696215 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64f559b798-5zt2v"] Jul 06 11:30:54.496205 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:54.496146 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:54.586284 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:54.586251 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:54.997921 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:54.997893 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6afc083-db02-461b-bd3e-65d6db51c5fd" path="/var/lib/kubelet/pods/e6afc083-db02-461b-bd3e-65d6db51c5fd/volumes" Jul 06 11:30:55.845008 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:55.844964 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:30:56.784436 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:56.784408 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:30:56.788333 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:30:56.788306 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-57df598f5-v2n47" Jul 06 11:31:01.582497 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:01.582465 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:01.682207 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:01.682179 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:28.128322 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:28.128279 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:28.190198 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:28.190147 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:32.426636 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.426605 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:31:32.427007 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.426890 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="alertmanager" containerID="cri-o://4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03" gracePeriod=120 Jul 06 11:31:32.427007 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.426963 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-metric" containerID="cri-o://9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69" gracePeriod=120 Jul 06 11:31:32.427152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.427000 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-web" containerID="cri-o://ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd" gracePeriod=120 Jul 06 11:31:32.427152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.427041 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy" containerID="cri-o://03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce" gracePeriod=120 Jul 06 11:31:32.427152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.427026 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="prom-label-proxy" containerID="cri-o://63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3" gracePeriod=120 Jul 06 11:31:32.427152 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.427021 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="config-reloader" containerID="cri-o://a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206" gracePeriod=120 Jul 06 11:31:32.823793 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823765 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3" exitCode=0 Jul 06 11:31:32.823793 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823788 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce" exitCode=0 Jul 06 11:31:32.823793 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823794 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206" exitCode=0 Jul 06 11:31:32.823793 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823801 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03" exitCode=0 Jul 06 11:31:32.824098 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823841 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3"} Jul 06 11:31:32.824098 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823923 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce"} Jul 06 11:31:32.824098 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.823963 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206"} Jul 06 11:31:32.824098 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:32.824005 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03"} Jul 06 11:31:33.675766 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.675735 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:33.775937 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.775912 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-config-out\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776097 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776023 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-main-tls\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776097 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776054 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-cluster-tls-config\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776097 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776084 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-web\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776292 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776117 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-metrics-client-ca\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776292 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776264 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-tls-assets\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776398 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776310 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-config-volume\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776398 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776347 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776398 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776382 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-web-config\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776420 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-main-db\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776472 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776487 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:31:33.776544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776521 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tln4q\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-kube-api-access-tln4q\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776760 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776555 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy\") pod \"ecff39fc-9b47-4622-aa57-4aa232f76764\" (UID: \"ecff39fc-9b47-4622-aa57-4aa232f76764\") " Jul 06 11:31:33.776840 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.776818 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-metrics-client-ca\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.778677 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.778643 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-config-out" (OuterVolumeSpecName: "config-out") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:31:33.778677 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.778647 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.779364 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.779120 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.779364 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.779240 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:31:33.779518 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.779434 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.779632 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.779603 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:31:33.779632 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.779615 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:31:33.779964 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.779942 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.780777 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.780758 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-kube-api-access-tln4q" (OuterVolumeSpecName: "kube-api-access-tln4q") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "kube-api-access-tln4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:31:33.780849 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.780759 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-config-volume" (OuterVolumeSpecName: "config-volume") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.783883 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.783851 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.789469 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.789448 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-web-config" (OuterVolumeSpecName: "web-config") pod "ecff39fc-9b47-4622-aa57-4aa232f76764" (UID: "ecff39fc-9b47-4622-aa57-4aa232f76764"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:31:33.831310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831283 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69" exitCode=0 Jul 06 11:31:33.831310 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831310 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerID="ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd" exitCode=0 Jul 06 11:31:33.831417 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831365 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69"} Jul 06 11:31:33.831453 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831440 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd"} Jul 06 11:31:33.831488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831377 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:33.831488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ecff39fc-9b47-4622-aa57-4aa232f76764","Type":"ContainerDied","Data":"9d4cad11d4d06364cff6cc7ebd0ae09deed6283f2e2f77a7bc439964109955e7"} Jul 06 11:31:33.831551 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.831507 2585 scope.go:117] "RemoveContainer" containerID="63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3" Jul 06 11:31:33.839022 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.839007 2585 scope.go:117] "RemoveContainer" containerID="9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69" Jul 06 11:31:33.845788 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.845770 2585 scope.go:117] "RemoveContainer" containerID="03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce" Jul 06 11:31:33.852543 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.852527 2585 scope.go:117] "RemoveContainer" containerID="ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd" Jul 06 11:31:33.858734 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.858718 2585 scope.go:117] "RemoveContainer" containerID="a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206" Jul 06 11:31:33.865237 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.865220 2585 scope.go:117] "RemoveContainer" containerID="4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03" Jul 06 11:31:33.872352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.872335 2585 scope.go:117] "RemoveContainer" containerID="8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e" Jul 06 11:31:33.873596 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.873580 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:31:33.878334 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878313 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878334 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878335 2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-web-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878346 2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-alertmanager-main-db\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878355 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878364 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tln4q\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-kube-api-access-tln4q\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878376 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878388 2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ecff39fc-9b47-4622-aa57-4aa232f76764-config-out\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878398 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-main-tls\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878405 2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-cluster-tls-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878414 2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878421 2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ecff39fc-9b47-4622-aa57-4aa232f76764-tls-assets\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.878480 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.878429 2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ecff39fc-9b47-4622-aa57-4aa232f76764-config-volume\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:31:33.879017 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.879001 2585 scope.go:117] "RemoveContainer" containerID="63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3" Jul 06 11:31:33.879347 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.879308 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3\": container with ID starting with 63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3 not found: ID does not exist" containerID="63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3" Jul 06 11:31:33.879452 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.879349 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3"} err="failed to get container status \"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3\": rpc error: code = NotFound desc = could not find container \"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3\": container with ID starting with 63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3 not found: ID does not exist" Jul 06 11:31:33.879452 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.879396 2585 scope.go:117] "RemoveContainer" containerID="9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69" Jul 06 11:31:33.879650 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.879634 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69\": container with ID starting with 9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69 not found: ID does not exist" containerID="9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69" Jul 06 11:31:33.879715 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.879658 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69"} err="failed to get container status \"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69\": rpc error: code = NotFound desc = could not find container \"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69\": container with ID starting with 9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69 not found: ID does not exist" Jul 06 11:31:33.879715 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.879679 2585 scope.go:117] "RemoveContainer" containerID="03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce" Jul 06 11:31:33.879992 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.879972 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce\": container with ID starting with 03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce not found: ID does not exist" containerID="03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce" Jul 06 11:31:33.880073 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.879997 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce"} err="failed to get container status \"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce\": rpc error: code = NotFound desc = could not find container \"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce\": container with ID starting with 03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce not found: ID does not exist" Jul 06 11:31:33.880073 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880012 2585 scope.go:117] "RemoveContainer" containerID="ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd" Jul 06 11:31:33.880314 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.880296 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd\": container with ID starting with ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd not found: ID does not exist" containerID="ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd" Jul 06 11:31:33.880359 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880318 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd"} err="failed to get container status \"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd\": rpc error: code = NotFound desc = could not find container \"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd\": container with ID starting with ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd not found: ID does not exist" Jul 06 11:31:33.880359 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880339 2585 scope.go:117] "RemoveContainer" containerID="a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206" Jul 06 11:31:33.880601 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.880586 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206\": container with ID starting with a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206 not found: ID does not exist" containerID="a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206" Jul 06 11:31:33.880665 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880611 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206"} err="failed to get container status \"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206\": rpc error: code = NotFound desc = could not find container \"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206\": container with ID starting with a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206 not found: ID does not exist" Jul 06 11:31:33.880665 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880624 2585 scope.go:117] "RemoveContainer" containerID="4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03" Jul 06 11:31:33.880870 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.880850 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03\": container with ID starting with 4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03 not found: ID does not exist" containerID="4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03" Jul 06 11:31:33.880913 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880875 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03"} err="failed to get container status \"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03\": rpc error: code = NotFound desc = could not find container \"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03\": container with ID starting with 4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03 not found: ID does not exist" Jul 06 11:31:33.880913 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.880890 2585 scope.go:117] "RemoveContainer" containerID="8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e" Jul 06 11:31:33.881125 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:31:33.881109 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e\": container with ID starting with 8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e not found: ID does not exist" containerID="8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e" Jul 06 11:31:33.881182 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881129 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e"} err="failed to get container status \"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e\": rpc error: code = NotFound desc = could not find container \"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e\": container with ID starting with 8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e not found: ID does not exist" Jul 06 11:31:33.881182 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881142 2585 scope.go:117] "RemoveContainer" containerID="63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3" Jul 06 11:31:33.881382 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881367 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:31:33.881457 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881388 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3"} err="failed to get container status \"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3\": rpc error: code = NotFound desc = could not find container \"63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3\": container with ID starting with 63ce0c93dbbf0f18e95d66809c0fc5e06d9d6f1f79d2aaaa9b15a03ba12690f3 not found: ID does not exist" Jul 06 11:31:33.881457 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881408 2585 scope.go:117] "RemoveContainer" containerID="9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69" Jul 06 11:31:33.881658 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881640 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69"} err="failed to get container status \"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69\": rpc error: code = NotFound desc = could not find container \"9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69\": container with ID starting with 9fdfb8061df21827eb8c826d871f2f85943dee0d11816ca1d1c3ed45e818fd69 not found: ID does not exist" Jul 06 11:31:33.881724 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881660 2585 scope.go:117] "RemoveContainer" containerID="03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce" Jul 06 11:31:33.881907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881889 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce"} err="failed to get container status \"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce\": rpc error: code = NotFound desc = could not find container \"03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce\": container with ID starting with 03de7fe77c1aadac729f32d3cced9feba868c54a1e7f205f5c3fa3f17cea80ce not found: ID does not exist" Jul 06 11:31:33.881961 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.881907 2585 scope.go:117] "RemoveContainer" containerID="ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd" Jul 06 11:31:33.882129 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882112 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd"} err="failed to get container status \"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd\": rpc error: code = NotFound desc = could not find container \"ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd\": container with ID starting with ffbfd30b9904851d37767503a9a9a61e7205b294c8b164a8e2dad2e60fe480cd not found: ID does not exist" Jul 06 11:31:33.882217 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882132 2585 scope.go:117] "RemoveContainer" containerID="a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206" Jul 06 11:31:33.882382 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882364 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206"} err="failed to get container status \"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206\": rpc error: code = NotFound desc = could not find container \"a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206\": container with ID starting with a477902a936338444adb3d30f7192f960c8f5ec35d5f2aaedca62619b642d206 not found: ID does not exist" Jul 06 11:31:33.882426 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882383 2585 scope.go:117] "RemoveContainer" containerID="4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03" Jul 06 11:31:33.882602 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882584 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03"} err="failed to get container status \"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03\": rpc error: code = NotFound desc = could not find container \"4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03\": container with ID starting with 4ef9b9f957d0566d35671bef38b05350b7ce663d455d74234abe76f07755ef03 not found: ID does not exist" Jul 06 11:31:33.882668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882604 2585 scope.go:117] "RemoveContainer" containerID="8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e" Jul 06 11:31:33.882793 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.882775 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e"} err="failed to get container status \"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e\": rpc error: code = NotFound desc = could not find container \"8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e\": container with ID starting with 8ead85c3695d5555e394ca8f56b0cd700fbeb0b2db488c5e6bdc26ae3f7e988e not found: ID does not exist" Jul 06 11:31:33.923136 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923109 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:31:33.923428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923414 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6afc083-db02-461b-bd3e-65d6db51c5fd" containerName="console" Jul 06 11:31:33.923470 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923458 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6afc083-db02-461b-bd3e-65d6db51c5fd" containerName="console" Jul 06 11:31:33.923504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923473 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-web" Jul 06 11:31:33.923504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923479 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-web" Jul 06 11:31:33.923504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923487 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-metric" Jul 06 11:31:33.923504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923493 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-metric" Jul 06 11:31:33.923504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923499 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="prom-label-proxy" Jul 06 11:31:33.923504 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923504 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="prom-label-proxy" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923520 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="config-reloader" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923525 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="config-reloader" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923532 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="init-config-reloader" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923537 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="init-config-reloader" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923543 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="alertmanager" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923548 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="alertmanager" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923557 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923562 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923611 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923618 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-web" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923623 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="prom-label-proxy" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923629 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="alertmanager" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923634 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="kube-rbac-proxy-metric" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923641 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" containerName="config-reloader" Jul 06 11:31:33.923668 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.923647 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6afc083-db02-461b-bd3e-65d6db51c5fd" containerName="console" Jul 06 11:31:33.928600 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.928585 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:33.931589 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.931572 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 11:31:33.931682 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.931613 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 11:31:33.931682 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.931667 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 11:31:33.932524 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.932510 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 11:31:33.933230 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.933217 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 11:31:33.933432 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.933419 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 11:31:33.933862 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.933844 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 11:31:33.937739 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.935753 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mt692\"" Jul 06 11:31:33.937739 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.936378 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 11:31:33.941497 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.941481 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 11:31:33.948011 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:33.947992 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:31:34.080062 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080005 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23b8c1ff-528e-48c3-a99b-d245b95612ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080062 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080038 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080265 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080068 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23b8c1ff-528e-48c3-a99b-d245b95612ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080265 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080087 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080265 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080138 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-web-config\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080265 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23b8c1ff-528e-48c3-a99b-d245b95612ca-config-out\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080467 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080467 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080313 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080467 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080363 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080467 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080396 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080467 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqsp\" (UniqueName: \"kubernetes.io/projected/23b8c1ff-528e-48c3-a99b-d245b95612ca-kube-api-access-zzqsp\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080618 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080466 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23b8c1ff-528e-48c3-a99b-d245b95612ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.080618 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.080504 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23b8c1ff-528e-48c3-a99b-d245b95612ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181252 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181228 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23b8c1ff-528e-48c3-a99b-d245b95612ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181266 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-web-config\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181350 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181299 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23b8c1ff-528e-48c3-a99b-d245b95612ca-config-out\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181498 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181367 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181498 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181426 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181784 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181763 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqsp\" (UniqueName: \"kubernetes.io/projected/23b8c1ff-528e-48c3-a99b-d245b95612ca-kube-api-access-zzqsp\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23b8c1ff-528e-48c3-a99b-d245b95612ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.181894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181887 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23b8c1ff-528e-48c3-a99b-d245b95612ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.182099 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23b8c1ff-528e-48c3-a99b-d245b95612ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.182099 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.181964 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184261 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184152 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/23b8c1ff-528e-48c3-a99b-d245b95612ca-config-out\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184721 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184459 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184721 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184524 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-web-config\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184721 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184533 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184721 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184556 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/23b8c1ff-528e-48c3-a99b-d245b95612ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184721 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184661 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/23b8c1ff-528e-48c3-a99b-d245b95612ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.184721 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184695 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.185124 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.184812 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.185210 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.185192 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23b8c1ff-528e-48c3-a99b-d245b95612ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.185275 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.185207 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.185327 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.185286 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23b8c1ff-528e-48c3-a99b-d245b95612ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.186402 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.186385 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/23b8c1ff-528e-48c3-a99b-d245b95612ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.191539 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.191520 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqsp\" (UniqueName: \"kubernetes.io/projected/23b8c1ff-528e-48c3-a99b-d245b95612ca-kube-api-access-zzqsp\") pod \"alertmanager-main-0\" (UID: \"23b8c1ff-528e-48c3-a99b-d245b95612ca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.240671 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.240643 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 11:31:34.373185 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.373138 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 11:31:34.375626 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:31:34.375603 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b8c1ff_528e_48c3_a99b_d245b95612ca.slice/crio-5b4d01c7e3d8bf9867262ee44185d8203f089e70daf7f2c1b261c29c4354c1b1 WatchSource:0}: Error finding container 5b4d01c7e3d8bf9867262ee44185d8203f089e70daf7f2c1b261c29c4354c1b1: Status 404 returned error can't find the container with id 5b4d01c7e3d8bf9867262ee44185d8203f089e70daf7f2c1b261c29c4354c1b1 Jul 06 11:31:34.531376 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.531353 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69cf7c7574-tmrh6"] Jul 06 11:31:34.534599 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.534583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.545835 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.545815 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cf7c7574-tmrh6"] Jul 06 11:31:34.592375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.592349 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:34.661792 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.661741 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:34.686017 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.685996 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-oauth-serving-cert\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.686330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.686021 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-config\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.686330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.686057 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-trusted-ca-bundle\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.686330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.686074 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-service-ca\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.686330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.686142 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p82mz\" (UniqueName: \"kubernetes.io/projected/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-kube-api-access-p82mz\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.686330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.686193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-oauth-config\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.686330 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.686223 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-serving-cert\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.726047 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.726025 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:34.787270 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787248 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-oauth-serving-cert\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.787397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787275 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-config\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.787397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-trusted-ca-bundle\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.787506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-service-ca\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.787506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787502 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p82mz\" (UniqueName: \"kubernetes.io/projected/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-kube-api-access-p82mz\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.787608 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787524 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-oauth-config\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.787608 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.787549 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-serving-cert\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.788028 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.788008 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-config\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.788135 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.788071 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-oauth-serving-cert\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.788219 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.788197 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-service-ca\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.788259 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.788219 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-trusted-ca-bundle\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.789999 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.789969 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-oauth-config\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.790221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.790202 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-serving-cert\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.795205 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.795187 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p82mz\" (UniqueName: \"kubernetes.io/projected/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-kube-api-access-p82mz\") pod \"console-69cf7c7574-tmrh6\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.827346 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.827325 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:34.836449 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.836426 2585 generic.go:356] "Generic (PLEG): container finished" podID="23b8c1ff-528e-48c3-a99b-d245b95612ca" containerID="cebbb0d02741cb04706998116bb9f590c94a9345486375f85a95ae414c1aca26" exitCode=0 Jul 06 11:31:34.836524 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.836512 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerDied","Data":"cebbb0d02741cb04706998116bb9f590c94a9345486375f85a95ae414c1aca26"} Jul 06 11:31:34.836606 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.836565 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"5b4d01c7e3d8bf9867262ee44185d8203f089e70daf7f2c1b261c29c4354c1b1"} Jul 06 11:31:34.843679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.843657 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:34.989586 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.989363 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cf7c7574-tmrh6"] Jul 06 11:31:34.998339 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:34.998313 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecff39fc-9b47-4622-aa57-4aa232f76764" path="/var/lib/kubelet/pods/ecff39fc-9b47-4622-aa57-4aa232f76764/volumes" Jul 06 11:31:35.004118 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:31:35.004098 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca5d2f4_bc0c_4ad5_9608_07ef5c3d4161.slice/crio-8b813a124eed3cf2402379d0447be592ef7e9095ceea25acaef3b421348ae8af WatchSource:0}: Error finding container 8b813a124eed3cf2402379d0447be592ef7e9095ceea25acaef3b421348ae8af: Status 404 returned error can't find the container with id 8b813a124eed3cf2402379d0447be592ef7e9095ceea25acaef3b421348ae8af Jul 06 11:31:35.324200 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.324152 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:35.381421 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.381397 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:35.843709 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.843678 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf7c7574-tmrh6" event={"ID":"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161","Type":"ContainerStarted","Data":"68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640"} Jul 06 11:31:35.844049 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.843735 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf7c7574-tmrh6" event={"ID":"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161","Type":"ContainerStarted","Data":"8b813a124eed3cf2402379d0447be592ef7e9095ceea25acaef3b421348ae8af"} Jul 06 11:31:35.847328 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.847303 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"e65fc4896fc2c5aded1d74eb35fd226cefdab6c2535966c201a4b9ce472d91cc"} Jul 06 11:31:35.847417 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.847365 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"f28f1d3fdd94884a143ad589277ac12f269a7623465200bacf46190c5ea1c813"} Jul 06 11:31:35.847417 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.847396 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"3161fba3512f795e201513f058c6ad231c41fc37270011cd45b4c7944a5b8357"} Jul 06 11:31:35.847487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.847423 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"e72612c66382ac97a7552d9d140cd50db1b794cbd6492f220a2be8de8f30b7ae"} Jul 06 11:31:35.847487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.847447 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"bb6d06f50e563a9d5dfa146d6975e3e02221f375057272964871a2e9f738fd0b"} Jul 06 11:31:35.847487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.847475 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"23b8c1ff-528e-48c3-a99b-d245b95612ca","Type":"ContainerStarted","Data":"88321ab6bf7dfceecf6f53bc528422556dea99b7d94a4e9f32be60716170a996"} Jul 06 11:31:35.862043 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.861994 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69cf7c7574-tmrh6" podStartSLOduration=1.861974427 podStartE2EDuration="1.861974427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:31:35.861430212 +0000 UTC m=+165.393169538" watchObservedRunningTime="2026-07-06 11:31:35.861974427 +0000 UTC m=+165.393713754" Jul 06 11:31:35.887629 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:35.887581 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.872092926 podStartE2EDuration="2.887561236s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.46831ms" isStatefulPod=false podCreationTimestamp="2026-07-06 11:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:31:35.88592449 +0000 UTC m=+165.417663816" watchObservedRunningTime="2026-07-06 11:31:35.887561236 +0000 UTC m=+165.419300564" Jul 06 11:31:39.998035 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:39.997990 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:40.099506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:40.099473 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:44.844101 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:44.844025 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:44.844101 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:44.844079 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:44.849413 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:44.849393 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:44.899540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:44.899518 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:31:44.960776 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:44.960751 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d9b4d489-rgdv2"] Jul 06 11:31:45.085359 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:45.085331 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:45.193759 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:45.193695 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:45.287963 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:45.287938 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:54.487852 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:54.487823 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:54.588741 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:54.588709 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:54.678736 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:54.678711 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:31:54.815980 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:31:54.815937 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:32:01.560718 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:01.560675 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:32:09.980952 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:09.980904 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58d9b4d489-rgdv2" podUID="62a211a7-b162-49b5-acb8-a3ca2fc49868" containerName="console" containerID="cri-o://e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851" gracePeriod=15 Jul 06 11:32:10.222436 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.222414 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:32:10.338385 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338358 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-config\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338394 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-serving-cert\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338413 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-service-ca\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338448 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-oauth-serving-cert\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338494 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-trusted-ca-bundle\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338676 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338516 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-oauth-config\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338676 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338553 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84lxw\" (UniqueName: \"kubernetes.io/projected/62a211a7-b162-49b5-acb8-a3ca2fc49868-kube-api-access-84lxw\") pod \"62a211a7-b162-49b5-acb8-a3ca2fc49868\" (UID: \"62a211a7-b162-49b5-acb8-a3ca2fc49868\") " Jul 06 11:32:10.338776 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338743 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-config" (OuterVolumeSpecName: "console-config") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:32:10.338966 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.338941 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:32:10.339144 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.339115 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:32:10.339279 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.339208 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-service-ca" (OuterVolumeSpecName: "service-ca") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:32:10.340535 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.340510 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:32:10.340676 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.340649 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:32:10.340722 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.340667 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62a211a7-b162-49b5-acb8-a3ca2fc49868-kube-api-access-84lxw" (OuterVolumeSpecName: "kube-api-access-84lxw") pod "62a211a7-b162-49b5-acb8-a3ca2fc49868" (UID: "62a211a7-b162-49b5-acb8-a3ca2fc49868"). InnerVolumeSpecName "kube-api-access-84lxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:32:10.439263 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439240 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-oauth-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:10.439263 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439259 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84lxw\" (UniqueName: \"kubernetes.io/projected/62a211a7-b162-49b5-acb8-a3ca2fc49868-kube-api-access-84lxw\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:10.439375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439268 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:10.439375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439278 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62a211a7-b162-49b5-acb8-a3ca2fc49868-console-serving-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:10.439375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439286 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-service-ca\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:10.439375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439294 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-oauth-serving-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:10.439375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:10.439301 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62a211a7-b162-49b5-acb8-a3ca2fc49868-trusted-ca-bundle\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:32:11.001562 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.001524 2585 generic.go:356] "Generic (PLEG): container finished" podID="62a211a7-b162-49b5-acb8-a3ca2fc49868" containerID="e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851" exitCode=0 Jul 06 11:32:11.002017 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.001644 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9b4d489-rgdv2" Jul 06 11:32:11.002092 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.002051 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9b4d489-rgdv2" event={"ID":"62a211a7-b162-49b5-acb8-a3ca2fc49868","Type":"ContainerDied","Data":"e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851"} Jul 06 11:32:11.002092 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.002087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9b4d489-rgdv2" event={"ID":"62a211a7-b162-49b5-acb8-a3ca2fc49868","Type":"ContainerDied","Data":"9510ef7480c56754d07eb5115fc1f23378ba74ee768cd119d24a1ecf708f4483"} Jul 06 11:32:11.002228 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.002113 2585 scope.go:117] "RemoveContainer" containerID="e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851" Jul 06 11:32:11.011119 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.011101 2585 scope.go:117] "RemoveContainer" containerID="e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851" Jul 06 11:32:11.011416 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:32:11.011394 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851\": container with ID starting with e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851 not found: ID does not exist" containerID="e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851" Jul 06 11:32:11.011493 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.011428 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851"} err="failed to get container status \"e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851\": rpc error: code = NotFound desc = could not find container \"e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851\": container with ID starting with e6fd7c5193eba4c739f9d0f26b59122c4dbd0af5848c453787b594390b668851 not found: ID does not exist" Jul 06 11:32:11.029491 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.029464 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d9b4d489-rgdv2"] Jul 06 11:32:11.032636 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:11.032617 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58d9b4d489-rgdv2"] Jul 06 11:32:12.998663 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:12.998615 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62a211a7-b162-49b5-acb8-a3ca2fc49868" path="/var/lib/kubelet/pods/62a211a7-b162-49b5-acb8-a3ca2fc49868/volumes" Jul 06 11:32:18.493389 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:18.493337 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:32:31.776025 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:31.775996 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:32:31.855419 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:31.855389 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:32:46.567975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:46.567943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:32:46.660642 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:32:46.660618 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:33:50.824073 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:33:50.824044 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 11:34:02.529121 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:02.529090 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:04.470526 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:04.470486 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:04.531387 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:04.531352 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:22.639952 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:22.639921 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:24.028735 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:24.028702 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:26.240179 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:26.240137 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:26.301356 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:26.301327 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:27.719750 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:27.719720 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:27.828184 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:27.828135 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:28.275171 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:28.275141 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:28.385268 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:28.385243 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:30.517111 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:30.517086 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:30.725541 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:30.725510 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:33.121372 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:33.121330 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:33.276451 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:33.276419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:36.344733 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.344697 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:36.589818 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.589787 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr"] Jul 06 11:34:36.590123 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.590111 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62a211a7-b162-49b5-acb8-a3ca2fc49868" containerName="console" Jul 06 11:34:36.590123 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.590124 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="62a211a7-b162-49b5-acb8-a3ca2fc49868" containerName="console" Jul 06 11:34:36.590258 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.590191 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="62a211a7-b162-49b5-acb8-a3ca2fc49868" containerName="console" Jul 06 11:34:36.593051 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.593031 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.595211 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.595151 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 11:34:36.595211 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.595151 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 11:34:36.595571 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.595554 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-j6cdl\"" Jul 06 11:34:36.602969 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.602951 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr"] Jul 06 11:34:36.659773 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.659749 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gffgz\" (UniqueName: \"kubernetes.io/projected/9808d04a-8715-4d0b-b97e-4a2fc5e31194-kube-api-access-gffgz\") pod \"cert-manager-cainjector-559cdd8c87-m6ndr\" (UID: \"9808d04a-8715-4d0b-b97e-4a2fc5e31194\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.659870 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.659778 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9808d04a-8715-4d0b-b97e-4a2fc5e31194-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m6ndr\" (UID: \"9808d04a-8715-4d0b-b97e-4a2fc5e31194\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.760757 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.760733 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gffgz\" (UniqueName: \"kubernetes.io/projected/9808d04a-8715-4d0b-b97e-4a2fc5e31194-kube-api-access-gffgz\") pod \"cert-manager-cainjector-559cdd8c87-m6ndr\" (UID: \"9808d04a-8715-4d0b-b97e-4a2fc5e31194\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.760847 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.760763 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9808d04a-8715-4d0b-b97e-4a2fc5e31194-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m6ndr\" (UID: \"9808d04a-8715-4d0b-b97e-4a2fc5e31194\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.775260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.775232 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gffgz\" (UniqueName: \"kubernetes.io/projected/9808d04a-8715-4d0b-b97e-4a2fc5e31194-kube-api-access-gffgz\") pod \"cert-manager-cainjector-559cdd8c87-m6ndr\" (UID: \"9808d04a-8715-4d0b-b97e-4a2fc5e31194\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.775418 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.775401 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9808d04a-8715-4d0b-b97e-4a2fc5e31194-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m6ndr\" (UID: \"9808d04a-8715-4d0b-b97e-4a2fc5e31194\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:36.902026 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:36.901972 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" Jul 06 11:34:37.028666 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:37.028567 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr"] Jul 06 11:34:37.031239 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:34:37.031212 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9808d04a_8715_4d0b_b97e_4a2fc5e31194.slice/crio-1b2209b588ad566064ac7071dd1f9aa0e9cafd2f507cc55a350c6b426b843995 WatchSource:0}: Error finding container 1b2209b588ad566064ac7071dd1f9aa0e9cafd2f507cc55a350c6b426b843995: Status 404 returned error can't find the container with id 1b2209b588ad566064ac7071dd1f9aa0e9cafd2f507cc55a350c6b426b843995 Jul 06 11:34:37.033223 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:37.033205 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 11:34:37.582513 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:37.582483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" event={"ID":"9808d04a-8715-4d0b-b97e-4a2fc5e31194","Type":"ContainerStarted","Data":"1b2209b588ad566064ac7071dd1f9aa0e9cafd2f507cc55a350c6b426b843995"} Jul 06 11:34:38.705704 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:38.705671 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:39.660276 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.660242 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9zxld"] Jul 06 11:34:39.664114 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.664087 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.666331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.666313 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6dpsj\"" Jul 06 11:34:39.674938 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.674911 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9zxld"] Jul 06 11:34:39.786104 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.786072 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc301c3a-64f7-4134-97bc-06ba53cda42f-bound-sa-token\") pod \"cert-manager-55b4d954d5-9zxld\" (UID: \"fc301c3a-64f7-4134-97bc-06ba53cda42f\") " pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.786602 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.786203 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pr7x\" (UniqueName: \"kubernetes.io/projected/fc301c3a-64f7-4134-97bc-06ba53cda42f-kube-api-access-6pr7x\") pod \"cert-manager-55b4d954d5-9zxld\" (UID: \"fc301c3a-64f7-4134-97bc-06ba53cda42f\") " pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.887232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.887202 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc301c3a-64f7-4134-97bc-06ba53cda42f-bound-sa-token\") pod \"cert-manager-55b4d954d5-9zxld\" (UID: \"fc301c3a-64f7-4134-97bc-06ba53cda42f\") " pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.887404 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.887305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pr7x\" (UniqueName: \"kubernetes.io/projected/fc301c3a-64f7-4134-97bc-06ba53cda42f-kube-api-access-6pr7x\") pod \"cert-manager-55b4d954d5-9zxld\" (UID: \"fc301c3a-64f7-4134-97bc-06ba53cda42f\") " pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.895938 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.895912 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc301c3a-64f7-4134-97bc-06ba53cda42f-bound-sa-token\") pod \"cert-manager-55b4d954d5-9zxld\" (UID: \"fc301c3a-64f7-4134-97bc-06ba53cda42f\") " pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.896058 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.896020 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pr7x\" (UniqueName: \"kubernetes.io/projected/fc301c3a-64f7-4134-97bc-06ba53cda42f-kube-api-access-6pr7x\") pod \"cert-manager-55b4d954d5-9zxld\" (UID: \"fc301c3a-64f7-4134-97bc-06ba53cda42f\") " pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:39.976056 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:39.975984 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9zxld" Jul 06 11:34:40.486586 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.486564 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9zxld"] Jul 06 11:34:40.488444 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:34:40.488418 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc301c3a_64f7_4134_97bc_06ba53cda42f.slice/crio-590943081bc1d5cd30349bc452dcf58c96ab39714bdcf7af5dc0b12b13848762 WatchSource:0}: Error finding container 590943081bc1d5cd30349bc452dcf58c96ab39714bdcf7af5dc0b12b13848762: Status 404 returned error can't find the container with id 590943081bc1d5cd30349bc452dcf58c96ab39714bdcf7af5dc0b12b13848762 Jul 06 11:34:40.597327 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.597287 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" event={"ID":"9808d04a-8715-4d0b-b97e-4a2fc5e31194","Type":"ContainerStarted","Data":"78dbc39b5b886f9753a64aa91a5000657f79f79790dd5c805e92ed46636462ab"} Jul 06 11:34:40.598821 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.598797 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9zxld" event={"ID":"fc301c3a-64f7-4134-97bc-06ba53cda42f","Type":"ContainerStarted","Data":"6d2e02b7a6cfdc0b9538ebf39be0fde25d7b590aed334ff4733e0db16c802947"} Jul 06 11:34:40.599013 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.598831 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9zxld" event={"ID":"fc301c3a-64f7-4134-97bc-06ba53cda42f","Type":"ContainerStarted","Data":"590943081bc1d5cd30349bc452dcf58c96ab39714bdcf7af5dc0b12b13848762"} Jul 06 11:34:40.614916 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.614872 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m6ndr" podStartSLOduration=1.219836232 podStartE2EDuration="4.61485596s" totalImagesPullingTime="3.395019728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:34:40.613768961 +0000 UTC m=+350.145508289" watchObservedRunningTime="2026-07-06 11:34:40.61485596 +0000 UTC m=+350.146595286" Jul 06 11:34:40.633271 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.633181 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9zxld" podStartSLOduration=1.633145996 podStartE2EDuration="1.633145996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:34:40.631705016 +0000 UTC m=+350.163444345" watchObservedRunningTime="2026-07-06 11:34:40.633145996 +0000 UTC m=+350.164885322" Jul 06 11:34:40.888717 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.888637 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:40.990553 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:40.990524 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:41.242386 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:41.242312 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:41.343100 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:41.343071 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:46.110632 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:46.110550 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:46.211396 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:46.211369 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:34:48.506813 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:34:48.506784 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.022265 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.022229 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.122028 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.121991 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.297303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.297244 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.526503 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.526479 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.626077 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.626018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.673810 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.673783 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg"] Jul 06 11:35:04.678931 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.678911 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.681418 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.681395 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 11:35:04.681526 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.681447 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dmxtp\"" Jul 06 11:35:04.681526 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.681477 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 11:35:04.681646 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.681570 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 11:35:04.681646 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.681615 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 11:35:04.691767 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.691746 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg"] Jul 06 11:35:04.774183 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.774126 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29dtx\" (UniqueName: \"kubernetes.io/projected/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-kube-api-access-29dtx\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.774341 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.774189 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-tmp\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.774341 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.774251 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-manifests\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.774341 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.774279 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.774481 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.774350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.807637 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.807613 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.875506 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.875480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29dtx\" (UniqueName: \"kubernetes.io/projected/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-kube-api-access-29dtx\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.875608 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.875522 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-tmp\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.875608 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.875551 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-manifests\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.875608 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.875580 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.875770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.875631 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.875977 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.875955 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-manifests\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.877892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.877837 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-tmp\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.877976 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.877957 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.878080 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.878062 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.884066 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.884047 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29dtx\" (UniqueName: \"kubernetes.io/projected/9393dbc6-a81e-402d-a3fe-5a2a05c97db8-kube-api-access-29dtx\") pod \"opendatahub-operator-controller-manager-6cb7946974-t5qpg\" (UID: \"9393dbc6-a81e-402d-a3fe-5a2a05c97db8\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:04.907791 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.907745 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:04.989658 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:04.989634 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:05.010153 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:05.010132 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:05.112773 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:05.112749 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:05.135487 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:05.135421 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg"] Jul 06 11:35:05.139460 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:35:05.139440 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9393dbc6_a81e_402d_a3fe_5a2a05c97db8.slice/crio-cf9eec2ae181d249ff7953058b6f89b00618877d947d5a151a3c1feb14f63323 WatchSource:0}: Error finding container cf9eec2ae181d249ff7953058b6f89b00618877d947d5a151a3c1feb14f63323: Status 404 returned error can't find the container with id cf9eec2ae181d249ff7953058b6f89b00618877d947d5a151a3c1feb14f63323 Jul 06 11:35:05.216040 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:05.215998 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:05.704919 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:05.704886 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" event={"ID":"9393dbc6-a81e-402d-a3fe-5a2a05c97db8","Type":"ContainerStarted","Data":"cf9eec2ae181d249ff7953058b6f89b00618877d947d5a151a3c1feb14f63323"} Jul 06 11:35:08.721260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:08.721231 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" event={"ID":"9393dbc6-a81e-402d-a3fe-5a2a05c97db8","Type":"ContainerStarted","Data":"04646b0f289f419760f6121f0a09066223b0271499b23864ece3293fed364b3a"} Jul 06 11:35:09.727868 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:09.727829 2585 generic.go:356] "Generic (PLEG): container finished" podID="9393dbc6-a81e-402d-a3fe-5a2a05c97db8" containerID="04646b0f289f419760f6121f0a09066223b0271499b23864ece3293fed364b3a" exitCode=0 Jul 06 11:35:09.728381 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:09.727874 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" event={"ID":"9393dbc6-a81e-402d-a3fe-5a2a05c97db8","Type":"ContainerDied","Data":"04646b0f289f419760f6121f0a09066223b0271499b23864ece3293fed364b3a"} Jul 06 11:35:10.735383 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:10.735304 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" event={"ID":"9393dbc6-a81e-402d-a3fe-5a2a05c97db8","Type":"ContainerStarted","Data":"b93d0aeee28e3e02bdf9c1604cd928af9ef9699c768d172d576d698cfe23ff84"} Jul 06 11:35:10.735778 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:10.735384 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:10.760852 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:10.760803 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" podStartSLOduration=2.052898713 podStartE2EDuration="6.760788466s" totalImagesPullingTime="3.647784474s" totalInitContainerRuntime="1.060105279s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:35:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:35:10.758840599 +0000 UTC m=+380.290579926" watchObservedRunningTime="2026-07-06 11:35:10.760788466 +0000 UTC m=+380.292527792" Jul 06 11:35:17.679497 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:17.679461 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:17.768125 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:17.768094 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:21.741556 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:21.741530 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cb7946974-t5qpg" Jul 06 11:35:25.084476 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:25.084441 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:25.209433 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:25.209407 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:25.863924 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:25.863890 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:25.964101 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:25.964072 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:26.069765 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:26.069724 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:26.465850 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:26.465828 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:26.769453 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:26.769422 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:28.553789 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.553754 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:28.625352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.625327 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64c57c544d-c2hml"] Jul 06 11:35:28.628729 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.628708 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.630767 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.630746 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 11:35:28.631031 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.631006 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 11:35:28.631204 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.631068 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 11:35:28.631204 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.631079 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 11:35:28.631204 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.631154 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nlx55\"" Jul 06 11:35:28.641848 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.641827 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64c57c544d-c2hml"] Jul 06 11:35:28.653322 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.653304 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tmp\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.653417 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.653350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tls-certs\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.653461 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.653432 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtwq2\" (UniqueName: \"kubernetes.io/projected/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-kube-api-access-xtwq2\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.753742 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.753723 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tls-certs\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.753853 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.753779 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwq2\" (UniqueName: \"kubernetes.io/projected/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-kube-api-access-xtwq2\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.753853 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.753826 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tmp\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.753853 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:35:28.753845 2585 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 06 11:35:28.754017 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:35:28.753902 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tls-certs podName:1bfd5cde-5fb9-40a0-b857-33193bee4a9d nodeName:}" failed. No retries permitted until 2026-07-06 11:35:29.253887216 +0000 UTC m=+398.785626520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tls-certs") pod "kube-auth-proxy-64c57c544d-c2hml" (UID: "1bfd5cde-5fb9-40a0-b857-33193bee4a9d") : secret "kube-auth-proxy-tls" not found Jul 06 11:35:28.756035 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.756016 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tmp\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:28.764128 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:28.764102 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwq2\" (UniqueName: \"kubernetes.io/projected/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-kube-api-access-xtwq2\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:29.011231 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.011151 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:29.257675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.257648 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tls-certs\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:29.259852 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.259834 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bfd5cde-5fb9-40a0-b857-33193bee4a9d-tls-certs\") pod \"kube-auth-proxy-64c57c544d-c2hml\" (UID: \"1bfd5cde-5fb9-40a0-b857-33193bee4a9d\") " pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:29.420116 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.420053 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6758ffdddb-fdq5v"] Jul 06 11:35:29.423545 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.423524 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.452525 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.452503 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6758ffdddb-fdq5v"] Jul 06 11:35:29.459074 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459053 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-oauth-serving-cert\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.459200 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459089 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-service-ca\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.459270 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459196 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sth5p\" (UniqueName: \"kubernetes.io/projected/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-kube-api-access-sth5p\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.459270 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459242 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-serving-cert\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.459379 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-oauth-config\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.459379 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459301 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-trusted-ca-bundle\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.459379 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.459340 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-config\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.538904 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.538880 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" Jul 06 11:35:29.540076 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.540052 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:29.560529 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560508 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-serving-cert\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.560859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-oauth-config\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.560859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560555 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-trusted-ca-bundle\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.560859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-config\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.560859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-oauth-serving-cert\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.560859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-service-ca\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.560859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.560709 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sth5p\" (UniqueName: \"kubernetes.io/projected/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-kube-api-access-sth5p\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.561358 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.561325 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-config\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.561725 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.561698 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-oauth-serving-cert\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.561842 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.561822 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-trusted-ca-bundle\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.561908 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.561893 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-service-ca\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.562975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.562953 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-oauth-config\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.563132 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.563113 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-console-serving-cert\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.575947 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.575919 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sth5p\" (UniqueName: \"kubernetes.io/projected/5e71f2f4-c9e8-43ec-a476-1da17b419f8c-kube-api-access-sth5p\") pod \"console-6758ffdddb-fdq5v\" (UID: \"5e71f2f4-c9e8-43ec-a476-1da17b419f8c\") " pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.623723 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.623693 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:29.672892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.672866 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64c57c544d-c2hml"] Jul 06 11:35:29.675503 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:35:29.675482 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfd5cde_5fb9_40a0_b857_33193bee4a9d.slice/crio-d60a3db7280a82410603f211f9a29401ea9cce5827128183ca7a111e1faf3a77 WatchSource:0}: Error finding container d60a3db7280a82410603f211f9a29401ea9cce5827128183ca7a111e1faf3a77: Status 404 returned error can't find the container with id d60a3db7280a82410603f211f9a29401ea9cce5827128183ca7a111e1faf3a77 Jul 06 11:35:29.733636 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.733614 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:29.821479 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.821442 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" event={"ID":"1bfd5cde-5fb9-40a0-b857-33193bee4a9d","Type":"ContainerStarted","Data":"d60a3db7280a82410603f211f9a29401ea9cce5827128183ca7a111e1faf3a77"} Jul 06 11:35:29.868726 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:29.868700 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6758ffdddb-fdq5v"] Jul 06 11:35:29.871564 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:35:29.871490 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e71f2f4_c9e8_43ec_a476_1da17b419f8c.slice/crio-4381e2cb7d1919f9a510264da621edef14290cdcad7d2b577f7f6b91340dcaa2 WatchSource:0}: Error finding container 4381e2cb7d1919f9a510264da621edef14290cdcad7d2b577f7f6b91340dcaa2: Status 404 returned error can't find the container with id 4381e2cb7d1919f9a510264da621edef14290cdcad7d2b577f7f6b91340dcaa2 Jul 06 11:35:30.012881 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:30.012853 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:30.107439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:30.107399 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:30.827662 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:30.827628 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6758ffdddb-fdq5v" event={"ID":"5e71f2f4-c9e8-43ec-a476-1da17b419f8c","Type":"ContainerStarted","Data":"8a3edc2c24a29dbbea36829828462511edec1a575989b2e9320232a2e98aa32b"} Jul 06 11:35:30.828132 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:30.827679 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6758ffdddb-fdq5v" event={"ID":"5e71f2f4-c9e8-43ec-a476-1da17b419f8c","Type":"ContainerStarted","Data":"4381e2cb7d1919f9a510264da621edef14290cdcad7d2b577f7f6b91340dcaa2"} Jul 06 11:35:30.847445 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:30.847403 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6758ffdddb-fdq5v" podStartSLOduration=1.847387231 podStartE2EDuration="1.847387231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:35:30.845472584 +0000 UTC m=+400.377211910" watchObservedRunningTime="2026-07-06 11:35:30.847387231 +0000 UTC m=+400.379126558" Jul 06 11:35:31.188940 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.188875 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:31.653440 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.653403 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh"] Jul 06 11:35:31.656817 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.656799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.658978 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.658957 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 11:35:31.659592 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.659568 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 11:35:31.659936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.659906 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 11:35:31.659936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.659929 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 11:35:31.660082 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.660018 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rcfnq\"" Jul 06 11:35:31.660140 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.660106 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 11:35:31.666203 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.666183 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh"] Jul 06 11:35:31.779995 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.779966 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-manager-config\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.780155 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.780003 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-metrics-cert\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.780155 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.780100 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-cert\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.780308 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.780177 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9h44\" (UniqueName: \"kubernetes.io/projected/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-kube-api-access-w9h44\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.881023 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.880989 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9h44\" (UniqueName: \"kubernetes.io/projected/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-kube-api-access-w9h44\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.881465 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.881117 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-manager-config\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.881465 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.881185 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-metrics-cert\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.881465 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.881438 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-cert\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.882155 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.882133 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-manager-config\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.883963 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.883940 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-metrics-cert\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.884087 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.884040 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-cert\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.889851 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.889833 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9h44\" (UniqueName: \"kubernetes.io/projected/c285df41-15e0-4fa0-8e3b-7a0ab50adc31-kube-api-access-w9h44\") pod \"lws-controller-manager-758cc6698f-2ljdh\" (UID: \"c285df41-15e0-4fa0-8e3b-7a0ab50adc31\") " pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:31.967130 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:31.967061 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:32.445232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:32.445211 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh"] Jul 06 11:35:32.447355 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:35:32.447328 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc285df41_15e0_4fa0_8e3b_7a0ab50adc31.slice/crio-ca70039b8ac0dd3296486ecb268c5036c322f76389007c8a96d122e6a760e18e WatchSource:0}: Error finding container ca70039b8ac0dd3296486ecb268c5036c322f76389007c8a96d122e6a760e18e: Status 404 returned error can't find the container with id ca70039b8ac0dd3296486ecb268c5036c322f76389007c8a96d122e6a760e18e Jul 06 11:35:32.840909 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:32.840877 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" event={"ID":"1bfd5cde-5fb9-40a0-b857-33193bee4a9d","Type":"ContainerStarted","Data":"44e5ae2e4d949385a4f7da145b7f7d83d449168359da38c3eda657228cf4f6ce"} Jul 06 11:35:32.841946 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:32.841925 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" event={"ID":"c285df41-15e0-4fa0-8e3b-7a0ab50adc31","Type":"ContainerStarted","Data":"ca70039b8ac0dd3296486ecb268c5036c322f76389007c8a96d122e6a760e18e"} Jul 06 11:35:32.860251 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:32.860210 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64c57c544d-c2hml" podStartSLOduration=2.165274826 podStartE2EDuration="4.860195749s" totalImagesPullingTime="2.694920923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:35:32.859114173 +0000 UTC m=+402.390853499" watchObservedRunningTime="2026-07-06 11:35:32.860195749 +0000 UTC m=+402.391935074" Jul 06 11:35:34.408416 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:34.408383 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:34.533892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:34.533856 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:35.579043 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:35.579011 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:35.759303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:35.759274 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:35.861111 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:35.861039 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" event={"ID":"c285df41-15e0-4fa0-8e3b-7a0ab50adc31","Type":"ContainerStarted","Data":"9e4a90b7e5ad05d616ce9df2eab1185ae3b221197509a3cc783d58b13e445997"} Jul 06 11:35:35.861111 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:35.861097 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:35:35.988965 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:35.988941 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:37.749491 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:37.749452 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:37.854733 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:37.854702 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:37.947549 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:37.947519 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:38.037511 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:38.037445 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:38.138434 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:38.138402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:38.237680 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:38.237655 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:38.325219 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:38.325129 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:38.925663 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:38.925627 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:39.734357 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:39.734324 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:39.734541 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:39.734378 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:39.739969 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:39.739942 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:39.760407 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:39.760362 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" podStartSLOduration=6.38895791 podStartE2EDuration="8.76034619s" totalImagesPullingTime="2.37138828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:35:35.916896553 +0000 UTC m=+405.448635880" watchObservedRunningTime="2026-07-06 11:35:39.76034619 +0000 UTC m=+409.292085517" Jul 06 11:35:39.884491 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:39.884466 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6758ffdddb-fdq5v" Jul 06 11:35:39.981521 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:39.981495 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cf7c7574-tmrh6"] Jul 06 11:35:40.180715 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:40.180675 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:40.307090 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:40.307058 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:40.366550 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:40.366526 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:35:46.869700 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:35:46.869664 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-758cc6698f-2ljdh" Jul 06 11:36:05.001284 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.001222 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-69cf7c7574-tmrh6" podUID="eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" containerName="console" containerID="cri-o://68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640" gracePeriod=15 Jul 06 11:36:05.242893 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.242870 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:36:05.347871 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.347806 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p82mz\" (UniqueName: \"kubernetes.io/projected/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-kube-api-access-p82mz\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.347871 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.347851 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-oauth-config\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.348014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.347889 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-config\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.348014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.347922 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-trusted-ca-bundle\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.348014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.347944 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-serving-cert\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.348014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.347976 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-oauth-serving-cert\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.348232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.348039 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-service-ca\") pod \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\" (UID: \"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161\") " Jul 06 11:36:05.348362 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.348333 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-config" (OuterVolumeSpecName: "console-config") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:36:05.348459 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.348359 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:36:05.348559 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.348448 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:36:05.348648 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.348623 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-service-ca" (OuterVolumeSpecName: "service-ca") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 11:36:05.350061 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.350036 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:36:05.350061 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.350050 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-kube-api-access-p82mz" (OuterVolumeSpecName: "kube-api-access-p82mz") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "kube-api-access-p82mz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:36:05.350188 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.350062 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" (UID: "eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:36:05.448619 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448597 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p82mz\" (UniqueName: \"kubernetes.io/projected/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-kube-api-access-p82mz\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:05.448619 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448617 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-oauth-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:05.448746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448627 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-config\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:05.448746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448637 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-trusted-ca-bundle\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:05.448746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448646 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-console-serving-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:05.448746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448654 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-oauth-serving-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:05.448746 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:05.448661 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161-service-ca\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:36:06.003271 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.003239 2585 generic.go:356] "Generic (PLEG): container finished" podID="eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" containerID="68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640" exitCode=0 Jul 06 11:36:06.003667 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.003281 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf7c7574-tmrh6" Jul 06 11:36:06.003667 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.003322 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf7c7574-tmrh6" event={"ID":"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161","Type":"ContainerDied","Data":"68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640"} Jul 06 11:36:06.003667 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.003368 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf7c7574-tmrh6" event={"ID":"eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161","Type":"ContainerDied","Data":"8b813a124eed3cf2402379d0447be592ef7e9095ceea25acaef3b421348ae8af"} Jul 06 11:36:06.003667 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.003391 2585 scope.go:117] "RemoveContainer" containerID="68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640" Jul 06 11:36:06.012602 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.012587 2585 scope.go:117] "RemoveContainer" containerID="68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640" Jul 06 11:36:06.012848 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:36:06.012826 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640\": container with ID starting with 68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640 not found: ID does not exist" containerID="68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640" Jul 06 11:36:06.012912 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.012861 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640"} err="failed to get container status \"68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640\": rpc error: code = NotFound desc = could not find container \"68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640\": container with ID starting with 68109d9e6600c70ae31888467f257b5bc1921ac8d5925f5f8cfc9d3d48626640 not found: ID does not exist" Jul 06 11:36:06.027407 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.027388 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cf7c7574-tmrh6"] Jul 06 11:36:06.030014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.029994 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69cf7c7574-tmrh6"] Jul 06 11:36:06.998983 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:06.998951 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" path="/var/lib/kubelet/pods/eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161/volumes" Jul 06 11:36:26.602015 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:26.601919 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:36:26.690329 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:26.690301 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:36:26.764116 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:26.764079 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:36:55.509428 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:36:55.509392 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:00.770130 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:00.770073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:01.591402 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:01.591365 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:03.703589 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:03.703557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:04.112809 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:04.112776 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:04.340255 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:04.340224 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:06.768052 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.768014 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pb92d"] Jul 06 11:37:06.768585 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.768565 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" containerName="console" Jul 06 11:37:06.768665 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.768588 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" containerName="console" Jul 06 11:37:06.768717 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.768665 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca5d2f4-bc0c-4ad5-9608-07ef5c3d4161" containerName="console" Jul 06 11:37:06.771083 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.771062 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:06.773444 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.773425 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 11:37:06.773868 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.773852 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 11:37:06.773968 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.773857 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4pvf4\"" Jul 06 11:37:06.782832 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.782811 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pb92d"] Jul 06 11:37:06.903407 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:06.903378 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9vsk\" (UniqueName: \"kubernetes.io/projected/f189420f-28d2-4950-bb3f-aad22c04a93c-kube-api-access-h9vsk\") pod \"authorino-operator-75cc8d5cb7-pb92d\" (UID: \"f189420f-28d2-4950-bb3f-aad22c04a93c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:07.004105 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.004082 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9vsk\" (UniqueName: \"kubernetes.io/projected/f189420f-28d2-4950-bb3f-aad22c04a93c-kube-api-access-h9vsk\") pod \"authorino-operator-75cc8d5cb7-pb92d\" (UID: \"f189420f-28d2-4950-bb3f-aad22c04a93c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:07.013077 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.013055 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9vsk\" (UniqueName: \"kubernetes.io/projected/f189420f-28d2-4950-bb3f-aad22c04a93c-kube-api-access-h9vsk\") pod \"authorino-operator-75cc8d5cb7-pb92d\" (UID: \"f189420f-28d2-4950-bb3f-aad22c04a93c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:07.082011 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.081964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:07.211319 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.211295 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pb92d"] Jul 06 11:37:07.213588 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:37:07.213559 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf189420f_28d2_4950_bb3f_aad22c04a93c.slice/crio-c01d10778ea89f51805122b8aa4ba98ad97b257f9bbd6580821d48e774159457 WatchSource:0}: Error finding container c01d10778ea89f51805122b8aa4ba98ad97b257f9bbd6580821d48e774159457: Status 404 returned error can't find the container with id c01d10778ea89f51805122b8aa4ba98ad97b257f9bbd6580821d48e774159457 Jul 06 11:37:07.287432 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.287404 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" event={"ID":"f189420f-28d2-4950-bb3f-aad22c04a93c","Type":"ContainerStarted","Data":"c01d10778ea89f51805122b8aa4ba98ad97b257f9bbd6580821d48e774159457"} Jul 06 11:37:07.307847 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.307813 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:07.540729 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:07.540693 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:08.757336 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:08.757297 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:09.010798 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:09.010730 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:10.305737 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:10.305691 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" event={"ID":"f189420f-28d2-4950-bb3f-aad22c04a93c","Type":"ContainerStarted","Data":"09b3d4c811a7b368266ab1dbc5484b8ed5cff258726fc58a01d4b80bfafb1788"} Jul 06 11:37:10.306194 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:10.305840 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:10.323203 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:10.323135 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" podStartSLOduration=2.139908428 podStartE2EDuration="4.323122042s" totalImagesPullingTime="2.183213614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:37:10.321836957 +0000 UTC m=+499.853576287" watchObservedRunningTime="2026-07-06 11:37:10.323122042 +0000 UTC m=+499.854861367" Jul 06 11:37:10.619546 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:10.619485 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:11.706650 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:11.706616 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:11.814532 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:11.814503 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:19.530556 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:19.530524 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:19.636575 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:19.636552 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:21.313868 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:21.313836 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pb92d" Jul 06 11:37:24.320262 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:24.320225 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:24.421213 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:24.421176 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:26.768176 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:26.768137 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:26.865760 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:26.865727 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:32.412297 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:32.412256 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:32.489489 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:32.489463 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:39.313795 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:39.313762 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:39.409697 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:39.409676 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:43.187738 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:43.187662 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:44.073007 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:44.072981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:44.174544 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:44.174517 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:44.573135 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:44.573107 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:44.694106 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:44.694081 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:46.652413 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:46.652373 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:46.743695 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:46.743663 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:47.244012 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:47.243984 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:47.379079 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:47.379053 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:52.157147 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:52.157114 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:52.259894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:52.259866 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:52.407101 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:52.407067 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:56.284868 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:56.284839 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:56.401990 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:56.401955 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:56.500142 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:56.500111 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:57.296677 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.296641 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hbrr8"] Jul 06 11:37:57.300093 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.300069 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:37:57.302481 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.302458 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fl6x7\"" Jul 06 11:37:57.308683 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.308661 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hbrr8"] Jul 06 11:37:57.334424 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.334401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:57.384618 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.384595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l8ws\" (UniqueName: \"kubernetes.io/projected/a5a4481d-38ec-4b98-977f-f71f058b7e23-kube-api-access-5l8ws\") pod \"authorino-58cd955675-hbrr8\" (UID: \"a5a4481d-38ec-4b98-977f-f71f058b7e23\") " pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:37:57.408986 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.408945 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:37:57.485491 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.485463 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8ws\" (UniqueName: \"kubernetes.io/projected/a5a4481d-38ec-4b98-977f-f71f058b7e23-kube-api-access-5l8ws\") pod \"authorino-58cd955675-hbrr8\" (UID: \"a5a4481d-38ec-4b98-977f-f71f058b7e23\") " pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:37:57.493488 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.493461 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8ws\" (UniqueName: \"kubernetes.io/projected/a5a4481d-38ec-4b98-977f-f71f058b7e23-kube-api-access-5l8ws\") pod \"authorino-58cd955675-hbrr8\" (UID: \"a5a4481d-38ec-4b98-977f-f71f058b7e23\") " pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:37:57.539911 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.539887 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-c7rjb"] Jul 06 11:37:57.543150 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.543134 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:37:57.549306 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.549257 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-c7rjb"] Jul 06 11:37:57.586197 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.586148 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s28lq\" (UniqueName: \"kubernetes.io/projected/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f-kube-api-access-s28lq\") pod \"authorino-75d556bd77-c7rjb\" (UID: \"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f\") " pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:37:57.610055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.610035 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:37:57.687499 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.687465 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s28lq\" (UniqueName: \"kubernetes.io/projected/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f-kube-api-access-s28lq\") pod \"authorino-75d556bd77-c7rjb\" (UID: \"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f\") " pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:37:57.697000 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.696969 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s28lq\" (UniqueName: \"kubernetes.io/projected/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f-kube-api-access-s28lq\") pod \"authorino-75d556bd77-c7rjb\" (UID: \"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f\") " pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:37:57.744834 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.744787 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hbrr8"] Jul 06 11:37:57.747597 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:37:57.747571 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5a4481d_38ec_4b98_977f_f71f058b7e23.slice/crio-182ad13fd3328c423fd289d562427bdc309092417002946f659a0bb400749e75 WatchSource:0}: Error finding container 182ad13fd3328c423fd289d562427bdc309092417002946f659a0bb400749e75: Status 404 returned error can't find the container with id 182ad13fd3328c423fd289d562427bdc309092417002946f659a0bb400749e75 Jul 06 11:37:57.853123 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.853060 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:37:57.988684 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:57.988632 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-c7rjb"] Jul 06 11:37:57.991398 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:37:57.991365 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8cc9d29_cbc6_4b86_94e1_c9b326498b7f.slice/crio-7a272381e601c8a95c5d35a55894017335f17bce6d4712f334933b9ad703763f WatchSource:0}: Error finding container 7a272381e601c8a95c5d35a55894017335f17bce6d4712f334933b9ad703763f: Status 404 returned error can't find the container with id 7a272381e601c8a95c5d35a55894017335f17bce6d4712f334933b9ad703763f Jul 06 11:37:58.543933 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:58.543885 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hbrr8" event={"ID":"a5a4481d-38ec-4b98-977f-f71f058b7e23","Type":"ContainerStarted","Data":"182ad13fd3328c423fd289d562427bdc309092417002946f659a0bb400749e75"} Jul 06 11:37:58.545469 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:37:58.545431 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-c7rjb" event={"ID":"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f","Type":"ContainerStarted","Data":"7a272381e601c8a95c5d35a55894017335f17bce6d4712f334933b9ad703763f"} Jul 06 11:38:00.073104 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:00.073072 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:00.173730 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:00.173697 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:01.565471 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:01.565430 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hbrr8" event={"ID":"a5a4481d-38ec-4b98-977f-f71f058b7e23","Type":"ContainerStarted","Data":"7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a"} Jul 06 11:38:01.567080 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:01.567049 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-c7rjb" event={"ID":"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f","Type":"ContainerStarted","Data":"9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3"} Jul 06 11:38:01.595686 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:01.595641 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hbrr8" podStartSLOduration=1.1163981139999999 podStartE2EDuration="4.595628057s" totalImagesPullingTime="3.479229943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:38:01.594350028 +0000 UTC m=+551.126089355" watchObservedRunningTime="2026-07-06 11:38:01.595628057 +0000 UTC m=+551.127367383" Jul 06 11:38:01.641582 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:01.641535 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-c7rjb" podStartSLOduration=1.395565814 podStartE2EDuration="4.641522078s" totalImagesPullingTime="3.245956264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:38:01.624272661 +0000 UTC m=+551.156011987" watchObservedRunningTime="2026-07-06 11:38:01.641522078 +0000 UTC m=+551.173261404" Jul 06 11:38:01.719510 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:01.719478 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hbrr8"] Jul 06 11:38:03.576084 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:03.576043 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hbrr8" podUID="a5a4481d-38ec-4b98-977f-f71f058b7e23" containerName="authorino" containerID="cri-o://7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a" gracePeriod=30 Jul 06 11:38:03.818639 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:03.818616 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:38:03.942198 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:03.942141 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l8ws\" (UniqueName: \"kubernetes.io/projected/a5a4481d-38ec-4b98-977f-f71f058b7e23-kube-api-access-5l8ws\") pod \"a5a4481d-38ec-4b98-977f-f71f058b7e23\" (UID: \"a5a4481d-38ec-4b98-977f-f71f058b7e23\") " Jul 06 11:38:03.944274 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:03.944249 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a4481d-38ec-4b98-977f-f71f058b7e23-kube-api-access-5l8ws" (OuterVolumeSpecName: "kube-api-access-5l8ws") pod "a5a4481d-38ec-4b98-977f-f71f058b7e23" (UID: "a5a4481d-38ec-4b98-977f-f71f058b7e23"). InnerVolumeSpecName "kube-api-access-5l8ws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:38:04.042828 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.042803 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l8ws\" (UniqueName: \"kubernetes.io/projected/a5a4481d-38ec-4b98-977f-f71f058b7e23-kube-api-access-5l8ws\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:38:04.484460 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.484425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:04.582203 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.582143 2585 generic.go:356] "Generic (PLEG): container finished" podID="a5a4481d-38ec-4b98-977f-f71f058b7e23" containerID="7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a" exitCode=0 Jul 06 11:38:04.582595 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.582212 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hbrr8" Jul 06 11:38:04.582595 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.582235 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hbrr8" event={"ID":"a5a4481d-38ec-4b98-977f-f71f058b7e23","Type":"ContainerDied","Data":"7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a"} Jul 06 11:38:04.582595 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.582286 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hbrr8" event={"ID":"a5a4481d-38ec-4b98-977f-f71f058b7e23","Type":"ContainerDied","Data":"182ad13fd3328c423fd289d562427bdc309092417002946f659a0bb400749e75"} Jul 06 11:38:04.582595 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.582310 2585 scope.go:117] "RemoveContainer" containerID="7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a" Jul 06 11:38:04.584328 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.584295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:04.591982 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.591964 2585 scope.go:117] "RemoveContainer" containerID="7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a" Jul 06 11:38:04.592281 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:38:04.592261 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a\": container with ID starting with 7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a not found: ID does not exist" containerID="7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a" Jul 06 11:38:04.592379 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.592286 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a"} err="failed to get container status \"7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a\": rpc error: code = NotFound desc = could not find container \"7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a\": container with ID starting with 7a932e55aa6b712e75b528b7104053fda3e169b95325412981f37ccd1112f95a not found: ID does not exist" Jul 06 11:38:04.602437 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.602413 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hbrr8"] Jul 06 11:38:04.607083 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.607062 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hbrr8"] Jul 06 11:38:04.998991 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:04.998962 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5a4481d-38ec-4b98-977f-f71f058b7e23" path="/var/lib/kubelet/pods/a5a4481d-38ec-4b98-977f-f71f058b7e23/volumes" Jul 06 11:38:06.941563 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:06.941529 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:07.290415 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:07.290382 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:17.837429 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:17.837395 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:29.608949 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.608915 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:29.843885 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.843855 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ksdgm"] Jul 06 11:38:29.844311 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.844292 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5a4481d-38ec-4b98-977f-f71f058b7e23" containerName="authorino" Jul 06 11:38:29.844412 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.844314 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5a4481d-38ec-4b98-977f-f71f058b7e23" containerName="authorino" Jul 06 11:38:29.844474 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.844418 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5a4481d-38ec-4b98-977f-f71f058b7e23" containerName="authorino" Jul 06 11:38:29.849005 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.848986 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:29.855178 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.855133 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ksdgm"] Jul 06 11:38:29.902940 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.902880 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:29.936617 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:29.936595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qggr9\" (UniqueName: \"kubernetes.io/projected/feaaceec-0ffd-4e3d-8e43-ace021ba3116-kube-api-access-qggr9\") pod \"authorino-5d4b67c446-ksdgm\" (UID: \"feaaceec-0ffd-4e3d-8e43-ace021ba3116\") " pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:30.037864 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.037840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qggr9\" (UniqueName: \"kubernetes.io/projected/feaaceec-0ffd-4e3d-8e43-ace021ba3116-kube-api-access-qggr9\") pod \"authorino-5d4b67c446-ksdgm\" (UID: \"feaaceec-0ffd-4e3d-8e43-ace021ba3116\") " pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:30.052425 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.052401 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qggr9\" (UniqueName: \"kubernetes.io/projected/feaaceec-0ffd-4e3d-8e43-ace021ba3116-kube-api-access-qggr9\") pod \"authorino-5d4b67c446-ksdgm\" (UID: \"feaaceec-0ffd-4e3d-8e43-ace021ba3116\") " pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:30.085630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.085608 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ksdgm"] Jul 06 11:38:30.085770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.085759 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:30.117658 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.117627 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f4478c586-vxgns"] Jul 06 11:38:30.120727 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.120713 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:30.128790 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.128765 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f4478c586-vxgns"] Jul 06 11:38:30.236089 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.236059 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ksdgm"] Jul 06 11:38:30.239261 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:38:30.239219 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfeaaceec_0ffd_4e3d_8e43_ace021ba3116.slice/crio-6556e483723d25c57bc9a7be8c7c714505bf9bea82049e86d1791f2b32e20919 WatchSource:0}: Error finding container 6556e483723d25c57bc9a7be8c7c714505bf9bea82049e86d1791f2b32e20919: Status 404 returned error can't find the container with id 6556e483723d25c57bc9a7be8c7c714505bf9bea82049e86d1791f2b32e20919 Jul 06 11:38:30.240563 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.240544 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw8dh\" (UniqueName: \"kubernetes.io/projected/c733f567-c783-4da7-bf6f-4ed39a12023a-kube-api-access-zw8dh\") pod \"authorino-6f4478c586-vxgns\" (UID: \"c733f567-c783-4da7-bf6f-4ed39a12023a\") " pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:30.245690 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.245671 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f4478c586-vxgns"] Jul 06 11:38:30.245891 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:38:30.245875 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zw8dh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f4478c586-vxgns" podUID="c733f567-c783-4da7-bf6f-4ed39a12023a" Jul 06 11:38:30.319587 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.319558 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b89968654-mcdfn"] Jul 06 11:38:30.324134 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.324112 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.327020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.327000 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 11:38:30.333608 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.333588 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b89968654-mcdfn"] Jul 06 11:38:30.341558 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.341537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw8dh\" (UniqueName: \"kubernetes.io/projected/c733f567-c783-4da7-bf6f-4ed39a12023a-kube-api-access-zw8dh\") pod \"authorino-6f4478c586-vxgns\" (UID: \"c733f567-c783-4da7-bf6f-4ed39a12023a\") " pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:30.358110 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.358086 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw8dh\" (UniqueName: \"kubernetes.io/projected/c733f567-c783-4da7-bf6f-4ed39a12023a-kube-api-access-zw8dh\") pod \"authorino-6f4478c586-vxgns\" (UID: \"c733f567-c783-4da7-bf6f-4ed39a12023a\") " pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:30.442303 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.442246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v488h\" (UniqueName: \"kubernetes.io/projected/772c2e15-faa8-4c23-88b1-a73d748e9583-kube-api-access-v488h\") pod \"authorino-b89968654-mcdfn\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.442402 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.442321 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/772c2e15-faa8-4c23-88b1-a73d748e9583-tls-cert\") pod \"authorino-b89968654-mcdfn\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.543609 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.543586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v488h\" (UniqueName: \"kubernetes.io/projected/772c2e15-faa8-4c23-88b1-a73d748e9583-kube-api-access-v488h\") pod \"authorino-b89968654-mcdfn\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.543688 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.543659 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/772c2e15-faa8-4c23-88b1-a73d748e9583-tls-cert\") pod \"authorino-b89968654-mcdfn\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.545804 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.545781 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/772c2e15-faa8-4c23-88b1-a73d748e9583-tls-cert\") pod \"authorino-b89968654-mcdfn\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.551763 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.551745 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v488h\" (UniqueName: \"kubernetes.io/projected/772c2e15-faa8-4c23-88b1-a73d748e9583-kube-api-access-v488h\") pod \"authorino-b89968654-mcdfn\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.633607 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.633589 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:38:30.711485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.711267 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" event={"ID":"feaaceec-0ffd-4e3d-8e43-ace021ba3116","Type":"ContainerStarted","Data":"5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee"} Jul 06 11:38:30.711485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.711284 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" podUID="feaaceec-0ffd-4e3d-8e43-ace021ba3116" containerName="authorino" containerID="cri-o://5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee" gracePeriod=30 Jul 06 11:38:30.711485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.711321 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" event={"ID":"feaaceec-0ffd-4e3d-8e43-ace021ba3116","Type":"ContainerStarted","Data":"6556e483723d25c57bc9a7be8c7c714505bf9bea82049e86d1791f2b32e20919"} Jul 06 11:38:30.711485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.711324 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:30.718590 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.718569 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:30.727869 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.727820 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" podStartSLOduration=1.351060106 podStartE2EDuration="1.727806721s" totalImagesPullingTime="376.746615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:38:30.727092262 +0000 UTC m=+580.258831589" watchObservedRunningTime="2026-07-06 11:38:30.727806721 +0000 UTC m=+580.259546046" Jul 06 11:38:30.776386 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.776362 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b89968654-mcdfn"] Jul 06 11:38:30.778830 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:38:30.778804 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod772c2e15_faa8_4c23_88b1_a73d748e9583.slice/crio-20c5d47b085ce4f97801d2296bb351dfd7165f89f286ad8ba35b9691ca03909e WatchSource:0}: Error finding container 20c5d47b085ce4f97801d2296bb351dfd7165f89f286ad8ba35b9691ca03909e: Status 404 returned error can't find the container with id 20c5d47b085ce4f97801d2296bb351dfd7165f89f286ad8ba35b9691ca03909e Jul 06 11:38:30.846636 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.846613 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw8dh\" (UniqueName: \"kubernetes.io/projected/c733f567-c783-4da7-bf6f-4ed39a12023a-kube-api-access-zw8dh\") pod \"c733f567-c783-4da7-bf6f-4ed39a12023a\" (UID: \"c733f567-c783-4da7-bf6f-4ed39a12023a\") " Jul 06 11:38:30.848472 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.848449 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c733f567-c783-4da7-bf6f-4ed39a12023a-kube-api-access-zw8dh" (OuterVolumeSpecName: "kube-api-access-zw8dh") pod "c733f567-c783-4da7-bf6f-4ed39a12023a" (UID: "c733f567-c783-4da7-bf6f-4ed39a12023a"). InnerVolumeSpecName "kube-api-access-zw8dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:38:30.947422 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.947400 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw8dh\" (UniqueName: \"kubernetes.io/projected/c733f567-c783-4da7-bf6f-4ed39a12023a-kube-api-access-zw8dh\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:38:30.951268 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:30.951245 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:31.048040 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.048020 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qggr9\" (UniqueName: \"kubernetes.io/projected/feaaceec-0ffd-4e3d-8e43-ace021ba3116-kube-api-access-qggr9\") pod \"feaaceec-0ffd-4e3d-8e43-ace021ba3116\" (UID: \"feaaceec-0ffd-4e3d-8e43-ace021ba3116\") " Jul 06 11:38:31.049855 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.049832 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feaaceec-0ffd-4e3d-8e43-ace021ba3116-kube-api-access-qggr9" (OuterVolumeSpecName: "kube-api-access-qggr9") pod "feaaceec-0ffd-4e3d-8e43-ace021ba3116" (UID: "feaaceec-0ffd-4e3d-8e43-ace021ba3116"). InnerVolumeSpecName "kube-api-access-qggr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:38:31.148601 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.148579 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qggr9\" (UniqueName: \"kubernetes.io/projected/feaaceec-0ffd-4e3d-8e43-ace021ba3116-kube-api-access-qggr9\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:38:31.718019 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.717983 2585 generic.go:356] "Generic (PLEG): container finished" podID="feaaceec-0ffd-4e3d-8e43-ace021ba3116" containerID="5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee" exitCode=2 Jul 06 11:38:31.718674 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.718029 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" Jul 06 11:38:31.718674 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.718088 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" event={"ID":"feaaceec-0ffd-4e3d-8e43-ace021ba3116","Type":"ContainerDied","Data":"5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee"} Jul 06 11:38:31.718674 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.718140 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ksdgm" event={"ID":"feaaceec-0ffd-4e3d-8e43-ace021ba3116","Type":"ContainerDied","Data":"6556e483723d25c57bc9a7be8c7c714505bf9bea82049e86d1791f2b32e20919"} Jul 06 11:38:31.718674 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.718191 2585 scope.go:117] "RemoveContainer" containerID="5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee" Jul 06 11:38:31.720462 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.720436 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4478c586-vxgns" Jul 06 11:38:31.720606 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.720434 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b89968654-mcdfn" event={"ID":"772c2e15-faa8-4c23-88b1-a73d748e9583","Type":"ContainerStarted","Data":"b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae"} Jul 06 11:38:31.720606 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.720501 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b89968654-mcdfn" event={"ID":"772c2e15-faa8-4c23-88b1-a73d748e9583","Type":"ContainerStarted","Data":"20c5d47b085ce4f97801d2296bb351dfd7165f89f286ad8ba35b9691ca03909e"} Jul 06 11:38:31.728620 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.728602 2585 scope.go:117] "RemoveContainer" containerID="5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee" Jul 06 11:38:31.728924 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:38:31.728906 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee\": container with ID starting with 5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee not found: ID does not exist" containerID="5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee" Jul 06 11:38:31.728970 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.728931 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee"} err="failed to get container status \"5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee\": rpc error: code = NotFound desc = could not find container \"5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee\": container with ID starting with 5533157bcea90842e98964c469b91ef8b1b497cfcd7e912132adbaa0295d74ee not found: ID does not exist" Jul 06 11:38:31.738061 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.738007 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b89968654-mcdfn" podStartSLOduration=1.404663143 podStartE2EDuration="1.737993371s" totalImagesPullingTime="333.330228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:38:31.736932376 +0000 UTC m=+581.268671703" watchObservedRunningTime="2026-07-06 11:38:31.737993371 +0000 UTC m=+581.269732696" Jul 06 11:38:31.763430 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.763403 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f4478c586-vxgns"] Jul 06 11:38:31.769623 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.769599 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-c7rjb"] Jul 06 11:38:31.769775 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.769756 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-c7rjb" podUID="b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" containerName="authorino" containerID="cri-o://9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3" gracePeriod=30 Jul 06 11:38:31.772352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.772329 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f4478c586-vxgns"] Jul 06 11:38:31.789314 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.789290 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ksdgm"] Jul 06 11:38:31.795988 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.795966 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ksdgm"] Jul 06 11:38:31.996291 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:31.996267 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:38:32.157977 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.157951 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s28lq\" (UniqueName: \"kubernetes.io/projected/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f-kube-api-access-s28lq\") pod \"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f\" (UID: \"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f\") " Jul 06 11:38:32.160055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.160028 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f-kube-api-access-s28lq" (OuterVolumeSpecName: "kube-api-access-s28lq") pod "b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" (UID: "b8cc9d29-cbc6-4b86-94e1-c9b326498b7f"). InnerVolumeSpecName "kube-api-access-s28lq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:38:32.258551 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.258525 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s28lq\" (UniqueName: \"kubernetes.io/projected/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f-kube-api-access-s28lq\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:38:32.727509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.727435 2585 generic.go:356] "Generic (PLEG): container finished" podID="b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" containerID="9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3" exitCode=0 Jul 06 11:38:32.727509 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.727482 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-c7rjb" Jul 06 11:38:32.728014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.727514 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-c7rjb" event={"ID":"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f","Type":"ContainerDied","Data":"9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3"} Jul 06 11:38:32.728014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.727557 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-c7rjb" event={"ID":"b8cc9d29-cbc6-4b86-94e1-c9b326498b7f","Type":"ContainerDied","Data":"7a272381e601c8a95c5d35a55894017335f17bce6d4712f334933b9ad703763f"} Jul 06 11:38:32.728014 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.727582 2585 scope.go:117] "RemoveContainer" containerID="9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3" Jul 06 11:38:32.735869 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.735851 2585 scope.go:117] "RemoveContainer" containerID="9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3" Jul 06 11:38:32.736092 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:38:32.736072 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3\": container with ID starting with 9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3 not found: ID does not exist" containerID="9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3" Jul 06 11:38:32.736185 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.736098 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3"} err="failed to get container status \"9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3\": rpc error: code = NotFound desc = could not find container \"9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3\": container with ID starting with 9633a491f741f79ff8cf9c981e4f00b7b5e1616feab7ecfeb274a896630b7de3 not found: ID does not exist" Jul 06 11:38:32.750049 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.750020 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-c7rjb"] Jul 06 11:38:32.757069 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.757048 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-c7rjb"] Jul 06 11:38:32.999919 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:32.999893 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" path="/var/lib/kubelet/pods/b8cc9d29-cbc6-4b86-94e1-c9b326498b7f/volumes" Jul 06 11:38:33.000215 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:33.000201 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c733f567-c783-4da7-bf6f-4ed39a12023a" path="/var/lib/kubelet/pods/c733f567-c783-4da7-bf6f-4ed39a12023a/volumes" Jul 06 11:38:33.000397 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:33.000386 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feaaceec-0ffd-4e3d-8e43-ace021ba3116" path="/var/lib/kubelet/pods/feaaceec-0ffd-4e3d-8e43-ace021ba3116/volumes" Jul 06 11:38:36.157975 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:36.157940 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:36.259894 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:36.259860 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:36.548625 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:36.548603 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:36.998124 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:36.998095 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:39.983507 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:39.983471 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:38:40.066963 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:38:40.066938 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:11.450575 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:11.450544 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:13.033193 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033094 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5gd74"] Jul 06 11:39:13.033540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033527 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feaaceec-0ffd-4e3d-8e43-ace021ba3116" containerName="authorino" Jul 06 11:39:13.033540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033541 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="feaaceec-0ffd-4e3d-8e43-ace021ba3116" containerName="authorino" Jul 06 11:39:13.033611 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033555 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" containerName="authorino" Jul 06 11:39:13.033611 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033561 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" containerName="authorino" Jul 06 11:39:13.033611 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033610 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="feaaceec-0ffd-4e3d-8e43-ace021ba3116" containerName="authorino" Jul 06 11:39:13.033834 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.033620 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8cc9d29-cbc6-4b86-94e1-c9b326498b7f" containerName="authorino" Jul 06 11:39:13.036508 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.036489 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.038618 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.038596 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pwtwt\"" Jul 06 11:39:13.038886 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.038871 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 11:39:13.049285 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.049260 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5gd74"] Jul 06 11:39:13.052921 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.052893 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b95f8b-e3e1-43ab-889f-7e17463c6a50-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5gd74\" (UID: \"78b95f8b-e3e1-43ab-889f-7e17463c6a50\") " pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.053013 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.052953 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srq8p\" (UniqueName: \"kubernetes.io/projected/78b95f8b-e3e1-43ab-889f-7e17463c6a50-kube-api-access-srq8p\") pod \"payload-pre-processing-7964457696-5gd74\" (UID: \"78b95f8b-e3e1-43ab-889f-7e17463c6a50\") " pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.153641 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.153616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b95f8b-e3e1-43ab-889f-7e17463c6a50-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5gd74\" (UID: \"78b95f8b-e3e1-43ab-889f-7e17463c6a50\") " pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.153752 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.153656 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srq8p\" (UniqueName: \"kubernetes.io/projected/78b95f8b-e3e1-43ab-889f-7e17463c6a50-kube-api-access-srq8p\") pod \"payload-pre-processing-7964457696-5gd74\" (UID: \"78b95f8b-e3e1-43ab-889f-7e17463c6a50\") " pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.154275 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.154257 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78b95f8b-e3e1-43ab-889f-7e17463c6a50-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5gd74\" (UID: \"78b95f8b-e3e1-43ab-889f-7e17463c6a50\") " pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.162238 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.162219 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srq8p\" (UniqueName: \"kubernetes.io/projected/78b95f8b-e3e1-43ab-889f-7e17463c6a50-kube-api-access-srq8p\") pod \"payload-pre-processing-7964457696-5gd74\" (UID: \"78b95f8b-e3e1-43ab-889f-7e17463c6a50\") " pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.191846 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.191814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:13.346041 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.345989 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:13.470917 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.470893 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5gd74"] Jul 06 11:39:13.473580 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:39:13.473552 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78b95f8b_e3e1_43ab_889f_7e17463c6a50.slice/crio-c06bc31be8dc6312fd1b3b69da9fae4561b51ef8984c164a327fce1cdcd51bad WatchSource:0}: Error finding container c06bc31be8dc6312fd1b3b69da9fae4561b51ef8984c164a327fce1cdcd51bad: Status 404 returned error can't find the container with id c06bc31be8dc6312fd1b3b69da9fae4561b51ef8984c164a327fce1cdcd51bad Jul 06 11:39:13.930861 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:13.930831 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" event={"ID":"78b95f8b-e3e1-43ab-889f-7e17463c6a50","Type":"ContainerStarted","Data":"c06bc31be8dc6312fd1b3b69da9fae4561b51ef8984c164a327fce1cdcd51bad"} Jul 06 11:39:14.943246 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:14.943204 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" event={"ID":"78b95f8b-e3e1-43ab-889f-7e17463c6a50","Type":"ContainerStarted","Data":"381822134cb3f45186e9cc66b2a2451143689e3448792e524a2dcdfbbac69bdb"} Jul 06 11:39:14.943597 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:14.943360 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:14.959973 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:14.959890 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" podStartSLOduration=0.729623428 podStartE2EDuration="1.959878951s" totalImagesPullingTime="1.230255523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:39:14.958128509 +0000 UTC m=+624.489867834" watchObservedRunningTime="2026-07-06 11:39:14.959878951 +0000 UTC m=+624.491618299" Jul 06 11:39:18.541866 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:18.541829 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:18.609907 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:18.609877 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:21.877532 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:21.877504 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:21.979800 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:21.979774 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:25.951084 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:25.951060 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-5gd74" Jul 06 11:39:29.536881 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:29.536846 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:43.849707 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:43.849668 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:51.087534 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.087499 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx"] Jul 06 11:39:51.092365 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.092342 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.094988 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.094965 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 11:39:51.095665 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.095648 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lkq6q\"" Jul 06 11:39:51.095770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.095649 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 11:39:51.095770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.095650 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 11:39:51.101439 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.101419 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx"] Jul 06 11:39:51.125055 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.125031 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:39:51.129264 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.129242 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bffkj\" (UniqueName: \"kubernetes.io/projected/189c68ff-cdd4-499a-a334-6e8c27c34cbe-kube-api-access-bffkj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.129366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.129306 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.129366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.129335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.129366 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.129362 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/189c68ff-cdd4-499a-a334-6e8c27c34cbe-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.129529 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.129394 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.129529 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.129452 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.230771 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.230746 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.230875 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.230807 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bffkj\" (UniqueName: \"kubernetes.io/projected/189c68ff-cdd4-499a-a334-6e8c27c34cbe-kube-api-access-bffkj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.230875 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.230847 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.230875 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.230862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.231103 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.230878 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/189c68ff-cdd4-499a-a334-6e8c27c34cbe-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.231103 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.230898 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.231235 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.231213 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.231291 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.231235 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.231329 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.231299 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.233039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.233016 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/189c68ff-cdd4-499a-a334-6e8c27c34cbe-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.233313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.233294 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/189c68ff-cdd4-499a-a334-6e8c27c34cbe-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.241642 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.241622 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bffkj\" (UniqueName: \"kubernetes.io/projected/189c68ff-cdd4-499a-a334-6e8c27c34cbe-kube-api-access-bffkj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q26hx\" (UID: \"189c68ff-cdd4-499a-a334-6e8c27c34cbe\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.402959 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.402896 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:39:51.534760 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.534735 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx"] Jul 06 11:39:51.536573 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:39:51.536548 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189c68ff_cdd4_499a_a334_6e8c27c34cbe.slice/crio-6a4f74ceace5151faaea9a365e5e215b3e5653160005b4a02b168bc98af11892 WatchSource:0}: Error finding container 6a4f74ceace5151faaea9a365e5e215b3e5653160005b4a02b168bc98af11892: Status 404 returned error can't find the container with id 6a4f74ceace5151faaea9a365e5e215b3e5653160005b4a02b168bc98af11892 Jul 06 11:39:51.537879 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:51.537865 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 11:39:52.122236 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:52.122114 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" event={"ID":"189c68ff-cdd4-499a-a334-6e8c27c34cbe","Type":"ContainerStarted","Data":"6a4f74ceace5151faaea9a365e5e215b3e5653160005b4a02b168bc98af11892"} Jul 06 11:39:53.128225 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:53.128193 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" event={"ID":"189c68ff-cdd4-499a-a334-6e8c27c34cbe","Type":"ContainerStarted","Data":"d4ec82c339534f6a67052c93be3962b2293d3e96e4dc55f566bc72e60967ed35"} Jul 06 11:39:53.148620 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:53.148563 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" podStartSLOduration=1.059312263 podStartE2EDuration="2.148546049s" totalImagesPullingTime="1.089233786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:39:53.146605433 +0000 UTC m=+662.678344771" watchObservedRunningTime="2026-07-06 11:39:53.148546049 +0000 UTC m=+662.680285374" Jul 06 11:39:56.426637 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:39:56.426606 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:01.403910 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:01.403877 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:40:01.404345 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:01.403937 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:40:01.416363 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:01.416333 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:40:02.187656 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:02.187627 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q26hx" Jul 06 11:40:13.787032 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:13.786996 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:19.449352 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:19.449321 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:26.008855 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:26.008821 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:26.108502 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:26.108476 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:36.832887 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:36.832853 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:54.197545 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.197453 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt"] Jul 06 11:40:54.201195 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.201170 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.203460 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.203432 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 11:40:54.211046 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.211017 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt"] Jul 06 11:40:54.307042 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.307014 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjr28\" (UniqueName: \"kubernetes.io/projected/7a0d7385-5906-4d37-8cc1-134728f45c16-kube-api-access-qjr28\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.307148 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.307064 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.307220 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.307156 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.307257 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.307218 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.307257 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.307237 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.307321 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.307263 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a0d7385-5906-4d37-8cc1-134728f45c16-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.319806 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.319784 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:40:54.407683 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.407654 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.407778 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.407687 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.407778 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.407713 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a0d7385-5906-4d37-8cc1-134728f45c16-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.407778 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.407752 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjr28\" (UniqueName: \"kubernetes.io/projected/7a0d7385-5906-4d37-8cc1-134728f45c16-kube-api-access-qjr28\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.407954 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.407935 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.408015 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.407980 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.408071 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.408058 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.408309 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.408284 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.408427 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.408307 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.409872 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.409843 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a0d7385-5906-4d37-8cc1-134728f45c16-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.410130 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.410113 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a0d7385-5906-4d37-8cc1-134728f45c16-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.415925 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.415905 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjr28\" (UniqueName: \"kubernetes.io/projected/7a0d7385-5906-4d37-8cc1-134728f45c16-kube-api-access-qjr28\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt\" (UID: \"7a0d7385-5906-4d37-8cc1-134728f45c16\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.510881 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.510864 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:40:54.643288 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:54.643264 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt"] Jul 06 11:40:54.645318 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:40:54.645291 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a0d7385_5906_4d37_8cc1_134728f45c16.slice/crio-c2703b9d983a814a4770eb6038db008f72db48d4ccf6cf23ec2b0545a27bcbbc WatchSource:0}: Error finding container c2703b9d983a814a4770eb6038db008f72db48d4ccf6cf23ec2b0545a27bcbbc: Status 404 returned error can't find the container with id c2703b9d983a814a4770eb6038db008f72db48d4ccf6cf23ec2b0545a27bcbbc Jul 06 11:40:55.430958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:55.430922 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" event={"ID":"7a0d7385-5906-4d37-8cc1-134728f45c16","Type":"ContainerStarted","Data":"8d2766dddac8b066b1481ee1cf06916da379a98c30d5235e49b502a30a1f45d4"} Jul 06 11:40:55.430958 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:55.430969 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" event={"ID":"7a0d7385-5906-4d37-8cc1-134728f45c16","Type":"ContainerStarted","Data":"c2703b9d983a814a4770eb6038db008f72db48d4ccf6cf23ec2b0545a27bcbbc"} Jul 06 11:40:55.449288 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:40:55.449235 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" podStartSLOduration=1.246000764 podStartE2EDuration="1.449219101s" totalImagesPullingTime="203.218337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:40:55.447653682 +0000 UTC m=+724.979393052" watchObservedRunningTime="2026-07-06 11:40:55.449219101 +0000 UTC m=+724.980958426" Jul 06 11:41:04.511829 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:04.511788 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:41:04.512309 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:04.511846 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:41:04.524468 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:04.524446 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:41:05.491094 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:05.491060 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt" Jul 06 11:41:29.408676 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:29.408643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:41:48.953365 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:48.953330 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8586b869f9-rdz4k"] Jul 06 11:41:48.956877 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:48.956854 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:48.970317 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:48.970295 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8586b869f9-rdz4k"] Jul 06 11:41:49.143260 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.143233 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84tgl\" (UniqueName: \"kubernetes.io/projected/e2a0c7b2-1f82-4b97-894e-c4e28cecf125-kube-api-access-84tgl\") pod \"authorino-8586b869f9-rdz4k\" (UID: \"e2a0c7b2-1f82-4b97-894e-c4e28cecf125\") " pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.143421 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.143294 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e2a0c7b2-1f82-4b97-894e-c4e28cecf125-tls-cert\") pod \"authorino-8586b869f9-rdz4k\" (UID: \"e2a0c7b2-1f82-4b97-894e-c4e28cecf125\") " pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.244686 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.244617 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84tgl\" (UniqueName: \"kubernetes.io/projected/e2a0c7b2-1f82-4b97-894e-c4e28cecf125-kube-api-access-84tgl\") pod \"authorino-8586b869f9-rdz4k\" (UID: \"e2a0c7b2-1f82-4b97-894e-c4e28cecf125\") " pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.244799 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.244692 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e2a0c7b2-1f82-4b97-894e-c4e28cecf125-tls-cert\") pod \"authorino-8586b869f9-rdz4k\" (UID: \"e2a0c7b2-1f82-4b97-894e-c4e28cecf125\") " pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.247006 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.246980 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e2a0c7b2-1f82-4b97-894e-c4e28cecf125-tls-cert\") pod \"authorino-8586b869f9-rdz4k\" (UID: \"e2a0c7b2-1f82-4b97-894e-c4e28cecf125\") " pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.253522 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.253500 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84tgl\" (UniqueName: \"kubernetes.io/projected/e2a0c7b2-1f82-4b97-894e-c4e28cecf125-kube-api-access-84tgl\") pod \"authorino-8586b869f9-rdz4k\" (UID: \"e2a0c7b2-1f82-4b97-894e-c4e28cecf125\") " pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.266247 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.266226 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8586b869f9-rdz4k" Jul 06 11:41:49.393853 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.393817 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8586b869f9-rdz4k"] Jul 06 11:41:49.396497 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:41:49.396474 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2a0c7b2_1f82_4b97_894e_c4e28cecf125.slice/crio-32a4edbafc22182c2160a579fbcb0f85fd64209223b4ac088c78ee4a04386065 WatchSource:0}: Error finding container 32a4edbafc22182c2160a579fbcb0f85fd64209223b4ac088c78ee4a04386065: Status 404 returned error can't find the container with id 32a4edbafc22182c2160a579fbcb0f85fd64209223b4ac088c78ee4a04386065 Jul 06 11:41:49.692019 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:49.691987 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8586b869f9-rdz4k" event={"ID":"e2a0c7b2-1f82-4b97-894e-c4e28cecf125","Type":"ContainerStarted","Data":"32a4edbafc22182c2160a579fbcb0f85fd64209223b4ac088c78ee4a04386065"} Jul 06 11:41:50.700921 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:50.700876 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8586b869f9-rdz4k" event={"ID":"e2a0c7b2-1f82-4b97-894e-c4e28cecf125","Type":"ContainerStarted","Data":"c98b4a4e25e6056ff976e3349cbee130c5d8dfa76a0397424391315399d692dc"} Jul 06 11:41:50.725221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:50.725134 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8586b869f9-rdz4k" podStartSLOduration=2.152058314 podStartE2EDuration="2.725116644s" totalImagesPullingTime="573.05833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:41:50.724313005 +0000 UTC m=+780.256052331" watchObservedRunningTime="2026-07-06 11:41:50.725116644 +0000 UTC m=+780.256855969" Jul 06 11:41:50.769984 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:50.769948 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b89968654-mcdfn"] Jul 06 11:41:50.770227 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:50.770196 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-b89968654-mcdfn" podUID="772c2e15-faa8-4c23-88b1-a73d748e9583" containerName="authorino" containerID="cri-o://b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae" gracePeriod=30 Jul 06 11:41:51.041003 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.040979 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:41:51.160154 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.160115 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v488h\" (UniqueName: \"kubernetes.io/projected/772c2e15-faa8-4c23-88b1-a73d748e9583-kube-api-access-v488h\") pod \"772c2e15-faa8-4c23-88b1-a73d748e9583\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " Jul 06 11:41:51.160338 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.160291 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/772c2e15-faa8-4c23-88b1-a73d748e9583-tls-cert\") pod \"772c2e15-faa8-4c23-88b1-a73d748e9583\" (UID: \"772c2e15-faa8-4c23-88b1-a73d748e9583\") " Jul 06 11:41:51.162230 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.162194 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772c2e15-faa8-4c23-88b1-a73d748e9583-kube-api-access-v488h" (OuterVolumeSpecName: "kube-api-access-v488h") pod "772c2e15-faa8-4c23-88b1-a73d748e9583" (UID: "772c2e15-faa8-4c23-88b1-a73d748e9583"). InnerVolumeSpecName "kube-api-access-v488h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:41:51.170941 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.170892 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772c2e15-faa8-4c23-88b1-a73d748e9583-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "772c2e15-faa8-4c23-88b1-a73d748e9583" (UID: "772c2e15-faa8-4c23-88b1-a73d748e9583"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:41:51.260956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.260928 2585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/772c2e15-faa8-4c23-88b1-a73d748e9583-tls-cert\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:41:51.260956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.260956 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v488h\" (UniqueName: \"kubernetes.io/projected/772c2e15-faa8-4c23-88b1-a73d748e9583-kube-api-access-v488h\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:41:51.706661 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.706570 2585 generic.go:356] "Generic (PLEG): container finished" podID="772c2e15-faa8-4c23-88b1-a73d748e9583" containerID="b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae" exitCode=0 Jul 06 11:41:51.706661 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.706624 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b89968654-mcdfn" Jul 06 11:41:51.707118 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.706653 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b89968654-mcdfn" event={"ID":"772c2e15-faa8-4c23-88b1-a73d748e9583","Type":"ContainerDied","Data":"b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae"} Jul 06 11:41:51.707118 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.706697 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b89968654-mcdfn" event={"ID":"772c2e15-faa8-4c23-88b1-a73d748e9583","Type":"ContainerDied","Data":"20c5d47b085ce4f97801d2296bb351dfd7165f89f286ad8ba35b9691ca03909e"} Jul 06 11:41:51.707118 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.706716 2585 scope.go:117] "RemoveContainer" containerID="b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae" Jul 06 11:41:51.716064 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.716048 2585 scope.go:117] "RemoveContainer" containerID="b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae" Jul 06 11:41:51.716364 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:41:51.716346 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae\": container with ID starting with b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae not found: ID does not exist" containerID="b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae" Jul 06 11:41:51.716429 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.716372 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae"} err="failed to get container status \"b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae\": rpc error: code = NotFound desc = could not find container \"b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae\": container with ID starting with b6c8ee92d35565f929759cf73b2774ecb5dd880890693eb78c404b7c9600a8ae not found: ID does not exist" Jul 06 11:41:51.730901 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.730875 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b89968654-mcdfn"] Jul 06 11:41:51.738119 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:51.738097 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b89968654-mcdfn"] Jul 06 11:41:52.998763 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:52.998731 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772c2e15-faa8-4c23-88b1-a73d748e9583" path="/var/lib/kubelet/pods/772c2e15-faa8-4c23-88b1-a73d748e9583/volumes" Jul 06 11:41:54.346447 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:54.346409 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:41:54.444712 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:54.444680 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:41:55.583039 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:55.583012 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:41:55.679122 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:41:55.679091 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:42:24.542624 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:42:24.542543 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:42:24.644012 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:42:24.643978 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:43:13.549620 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:43:13.549578 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:43:14.838218 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:43:14.838182 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:02.600272 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:02.600220 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:02.695422 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:02.695399 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:02.877199 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:02.877093 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:09.899184 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:09.899126 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:10.007053 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:10.007024 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:10.099486 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:10.099460 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:10.302970 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:10.302949 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:16.748985 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:16.748947 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:18.151214 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:18.151131 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:44:22.630377 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:44:22.630327 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:47:15.142540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:47:15.142446 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:48:19.936687 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:48:19.936618 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:48:20.034833 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:48:20.034809 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:48:58.224100 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:48:58.224053 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:49:34.845762 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:49:34.845728 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:51:32.998281 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:51:32.998195 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:06.626313 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:06.626253 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:07.705762 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:07.705730 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:12.904282 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:12.904247 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:12.962566 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:12.962538 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:13.167692 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:13.167616 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:13.267626 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:13.267602 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:14.298931 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:14.298902 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:14.371321 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:14.371289 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:14.436357 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:14.436332 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:14.822365 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:14.822336 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:15.037966 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:15.037937 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:15.137571 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:15.137508 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:17.859479 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:17.859444 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:19.447895 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:19.447865 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:19.615503 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:19.615467 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:19.715690 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:19.715621 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:19.870969 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:19.870940 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:21.060032 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:21.060004 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:25.689653 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:25.689624 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:25.792892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:25.792859 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:25.915867 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:25.915844 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:26.008848 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:26.008821 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:27.116539 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:27.116510 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:27.191804 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:27.191771 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:27.251761 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:27.251734 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:27.522684 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:27.522657 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:27.815117 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:27.815043 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:27.915922 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:27.915884 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:29.172318 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:29.172286 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:32.242916 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:32.242886 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:32.609010 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:32.608981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:38.564094 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:38.564063 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:38.666031 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:38.666001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:44.629847 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:44.629813 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:44.688510 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:44.688477 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:44.822155 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:44.822127 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:46.023667 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:46.023620 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:46.110301 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:46.110275 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:46.175349 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:46.175329 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:46.632740 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:46.632717 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:46.733103 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:46.733075 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:46.834056 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:46.834034 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:50.463099 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:50.463067 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:50.627667 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:50.627639 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:50.737068 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:50.736995 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:50.859317 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:50.859279 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:50.972258 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:50.972231 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:56.691882 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:56.691851 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:56.745675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:56.745646 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:57.227838 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:57.227806 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:57.339527 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:57.339498 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:57.433087 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:57.433066 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:55:58.747733 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:55:58.747703 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:04.908056 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:04.908025 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:05.004132 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:05.004107 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:05.079579 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:05.079551 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:05.175351 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:05.175286 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:05.283507 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:05.283478 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:05.624174 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:05.624140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:31.567868 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:31.567835 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:37.564827 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:37.564793 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:37.667859 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:37.667831 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:37.765285 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:37.765262 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:53.091983 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:53.091949 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:53.625869 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:53.625820 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:59.615124 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:59.615092 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:59.674403 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:59.674380 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:59.741638 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:59.741605 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:56:59.843331 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:56:59.843313 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:10.333388 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:10.333336 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:11.114009 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:11.113983 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:16.945599 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:16.945515 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:17.047847 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:17.047817 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:17.149471 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:17.149447 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:17.276745 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:17.276717 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:17.992803 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:17.992762 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:18.089041 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:18.089018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:18.187707 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:18.187683 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:19.764306 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:19.764277 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:25.666881 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:25.666837 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:25.767675 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:25.767643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:26.322178 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:26.322140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:32.126985 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:32.126948 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:32.196220 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:32.196189 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:32.358380 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:32.358348 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:32.679048 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:32.679012 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:32.772762 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:32.772731 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:35.854219 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:35.854189 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:35.934613 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:35.934583 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:36.004232 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:36.004206 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:36.362918 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:36.362890 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:36.462335 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:36.462303 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:36.563170 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:36.563143 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:40.306193 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:40.306132 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:40.938651 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:40.938622 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:41.049797 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:41.049772 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:41.382434 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:41.382413 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:47.338929 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:47.338892 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:47.397034 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:47.397008 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:47.733724 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:47.733655 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:48.063553 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:48.063526 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:48.161119 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:48.161093 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:53.846822 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:53.846776 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:53.906857 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:53.906831 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:54.644626 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:54.644596 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:54.745224 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:54.745190 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:54.945008 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:54.944934 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:55.136536 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:55.136504 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:55.251180 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:55.251130 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:55.356127 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:55.356106 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:57:55.469002 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:57:55.468978 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:01.436353 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:01.436316 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:01.499472 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:01.499448 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:01.574981 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:01.574952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:01.714191 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:01.714097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:01.806837 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:01.806817 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:07.741984 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:07.741946 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:07.864776 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:07.864746 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:14.532106 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:14.532071 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:14.898936 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:14.898843 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:15.167221 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:15.167134 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:18.019787 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:18.019749 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:18.099841 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:18.099805 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:18.201332 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:18.201301 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:18.679686 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:18.679661 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:18.780641 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:18.780620 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:22.498954 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:22.498917 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:33.418383 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.418353 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq"] Jul 06 11:58:33.418798 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.418721 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772c2e15-faa8-4c23-88b1-a73d748e9583" containerName="authorino" Jul 06 11:58:33.418798 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.418733 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="772c2e15-faa8-4c23-88b1-a73d748e9583" containerName="authorino" Jul 06 11:58:33.418882 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.418807 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="772c2e15-faa8-4c23-88b1-a73d748e9583" containerName="authorino" Jul 06 11:58:33.421916 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.421895 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.424022 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.424002 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b3137601\"/\"openshift-service-ca.crt\"" Jul 06 11:58:33.424616 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.424595 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b3137601\"/\"e2e-lifecycle-model-b3137601-kserve-self-signed-certs\"" Jul 06 11:58:33.424709 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.424603 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b3137601\"/\"kube-root-ca.crt\"" Jul 06 11:58:33.424771 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.424624 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b3137601\"/\"default-dockercfg-h6gw4\"" Jul 06 11:58:33.430988 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.430961 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq"] Jul 06 11:58:33.450207 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.450182 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:33.525298 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.525274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-home\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.525404 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.525300 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-dshm\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.525404 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.525322 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnvk6\" (UniqueName: \"kubernetes.io/projected/7d4d566c-a768-49ef-a8a7-c62f22563fb7-kube-api-access-rnvk6\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.525517 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.525425 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tls-certs\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.525517 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.525487 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tmp-dir\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.525620 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.525529 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-model-cache\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626082 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-home\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626208 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626087 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-dshm\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626208 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvk6\" (UniqueName: \"kubernetes.io/projected/7d4d566c-a768-49ef-a8a7-c62f22563fb7-kube-api-access-rnvk6\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626334 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tls-certs\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626334 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tmp-dir\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626334 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-model-cache\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626493 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626461 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-home\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626629 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626610 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-model-cache\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.626691 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.626675 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tmp-dir\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.628382 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.628363 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-dshm\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.628637 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.628620 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tls-certs\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.635934 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.635914 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnvk6\" (UniqueName: \"kubernetes.io/projected/7d4d566c-a768-49ef-a8a7-c62f22563fb7-kube-api-access-rnvk6\") pod \"e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.732409 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.732358 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:33.859573 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:33.859551 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq"] Jul 06 11:58:33.862329 ip-10-0-143-173 kubenswrapper[2585]: W0706 11:58:33.862292 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d4d566c_a768_49ef_a8a7_c62f22563fb7.slice/crio-3430c889697ae25f1ce21ce6277d0bc6515d599a55b33731d16f2866027a33ee WatchSource:0}: Error finding container 3430c889697ae25f1ce21ce6277d0bc6515d599a55b33731d16f2866027a33ee: Status 404 returned error can't find the container with id 3430c889697ae25f1ce21ce6277d0bc6515d599a55b33731d16f2866027a33ee Jul 06 11:58:34.565179 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:34.565133 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" event={"ID":"7d4d566c-a768-49ef-a8a7-c62f22563fb7","Type":"ContainerStarted","Data":"5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13"} Jul 06 11:58:34.565179 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:34.565184 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" event={"ID":"7d4d566c-a768-49ef-a8a7-c62f22563fb7","Type":"ContainerStarted","Data":"3430c889697ae25f1ce21ce6277d0bc6515d599a55b33731d16f2866027a33ee"} Jul 06 11:58:34.581375 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:34.581326 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" podStartSLOduration=1.581310506 podStartE2EDuration="1.581310506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 11:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 11:58:34.580528203 +0000 UTC m=+1784.112267528" watchObservedRunningTime="2026-07-06 11:58:34.581310506 +0000 UTC m=+1784.113049831" Jul 06 11:58:38.321540 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:38.321501 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:38.421199 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:38.421146 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:40.266815 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:40.266786 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:40.367359 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:40.367327 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:43.732987 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:43.732953 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:43.733415 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:43.733001 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:43.745682 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:43.745658 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:44.625516 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:44.625490 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:58:54.890679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:54.890645 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:58:55.213028 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:58:55.212947 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:00.928041 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:00.928003 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:05.924790 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:05.924745 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq"] Jul 06 11:59:06.009341 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:06.009311 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:20.945287 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:20.945234 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" podUID="7d4d566c-a768-49ef-a8a7-c62f22563fb7" containerName="main" containerID="cri-o://5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13" gracePeriod=45 Jul 06 11:59:21.184450 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.184427 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:59:21.304524 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304501 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-model-cache\") pod \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " Jul 06 11:59:21.304679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304532 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tls-certs\") pod \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " Jul 06 11:59:21.304679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304567 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnvk6\" (UniqueName: \"kubernetes.io/projected/7d4d566c-a768-49ef-a8a7-c62f22563fb7-kube-api-access-rnvk6\") pod \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " Jul 06 11:59:21.304679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304586 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-home\") pod \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " Jul 06 11:59:21.304679 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304605 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-dshm\") pod \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " Jul 06 11:59:21.304892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304738 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tmp-dir\") pod \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\" (UID: \"7d4d566c-a768-49ef-a8a7-c62f22563fb7\") " Jul 06 11:59:21.304892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304809 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7d4d566c-a768-49ef-a8a7-c62f22563fb7" (UID: "7d4d566c-a768-49ef-a8a7-c62f22563fb7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:59:21.304892 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304825 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-home" (OuterVolumeSpecName: "home") pod "7d4d566c-a768-49ef-a8a7-c62f22563fb7" (UID: "7d4d566c-a768-49ef-a8a7-c62f22563fb7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:59:21.305047 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.304959 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7d4d566c-a768-49ef-a8a7-c62f22563fb7" (UID: "7d4d566c-a768-49ef-a8a7-c62f22563fb7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:59:21.305114 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.305078 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:59:21.305114 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.305094 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:59:21.305114 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.305107 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:59:21.306725 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.306702 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7d4d566c-a768-49ef-a8a7-c62f22563fb7" (UID: "7d4d566c-a768-49ef-a8a7-c62f22563fb7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 11:59:21.306818 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.306737 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4d566c-a768-49ef-a8a7-c62f22563fb7-kube-api-access-rnvk6" (OuterVolumeSpecName: "kube-api-access-rnvk6") pod "7d4d566c-a768-49ef-a8a7-c62f22563fb7" (UID: "7d4d566c-a768-49ef-a8a7-c62f22563fb7"). InnerVolumeSpecName "kube-api-access-rnvk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 11:59:21.306818 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.306748 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-dshm" (OuterVolumeSpecName: "dshm") pod "7d4d566c-a768-49ef-a8a7-c62f22563fb7" (UID: "7d4d566c-a768-49ef-a8a7-c62f22563fb7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 11:59:21.405588 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.405567 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d4d566c-a768-49ef-a8a7-c62f22563fb7-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:59:21.405588 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.405588 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d4d566c-a768-49ef-a8a7-c62f22563fb7-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:59:21.405770 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.405598 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnvk6\" (UniqueName: \"kubernetes.io/projected/7d4d566c-a768-49ef-a8a7-c62f22563fb7-kube-api-access-rnvk6\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 11:59:21.802485 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.802450 2585 generic.go:356] "Generic (PLEG): container finished" podID="7d4d566c-a768-49ef-a8a7-c62f22563fb7" containerID="5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13" exitCode=0 Jul 06 11:59:21.802630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.802502 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" Jul 06 11:59:21.802630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.802543 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" event={"ID":"7d4d566c-a768-49ef-a8a7-c62f22563fb7","Type":"ContainerDied","Data":"5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13"} Jul 06 11:59:21.802630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.802587 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq" event={"ID":"7d4d566c-a768-49ef-a8a7-c62f22563fb7","Type":"ContainerDied","Data":"3430c889697ae25f1ce21ce6277d0bc6515d599a55b33731d16f2866027a33ee"} Jul 06 11:59:21.802630 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.802604 2585 scope.go:117] "RemoveContainer" containerID="5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13" Jul 06 11:59:21.811865 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.811844 2585 scope.go:117] "RemoveContainer" containerID="5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13" Jul 06 11:59:21.812130 ip-10-0-143-173 kubenswrapper[2585]: E0706 11:59:21.812112 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13\": container with ID starting with 5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13 not found: ID does not exist" containerID="5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13" Jul 06 11:59:21.812191 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.812139 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13"} err="failed to get container status \"5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13\": rpc error: code = NotFound desc = could not find container \"5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13\": container with ID starting with 5c150ae89f2083c97b6fbe3022fe5a7cbba7ce69347838eac1024c5f5046fd13 not found: ID does not exist" Jul 06 11:59:21.822829 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.822809 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq"] Jul 06 11:59:21.828217 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:21.828197 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b3137601/e2e-lifecycle-model-b3137601-kserve-6754c4fdbc-j55tq"] Jul 06 11:59:23.000822 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:23.000782 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d4d566c-a768-49ef-a8a7-c62f22563fb7" path="/var/lib/kubelet/pods/7d4d566c-a768-49ef-a8a7-c62f22563fb7/volumes" Jul 06 11:59:42.069437 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:42.069404 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:43.446020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:43.445984 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:49.156513 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:49.156476 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:49.255814 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:49.255787 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:49.450020 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:49.449945 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:49.673906 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:49.673877 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:50.734831 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:50.734804 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:50.843956 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:50.843933 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:50.945012 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:50.944988 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:51.632896 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:51.632867 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:51.757651 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:51.757627 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:51.857819 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:51.857796 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:53.145634 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:53.145602 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:59.075256 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:59.075228 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:59.135144 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:59.135115 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 11:59:59.727809 ip-10-0-143-173 kubenswrapper[2585]: I0706 11:59:59.727774 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:05.582847 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:05.582814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:05.706985 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:05.706958 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:06.244405 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:06.244372 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:16.274239 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:16.274152 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:20.594077 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:20.594039 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:38.299937 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:38.299894 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:43.472452 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:43.472416 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:43.530847 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:43.530822 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:43.808047 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:43.808021 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:44.112289 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:44.112223 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:44.615079 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:44.615028 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:47.074892 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:47.074860 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:47.177910 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:47.177885 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:47.727624 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:47.727595 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:47.827604 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:47.827574 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:51.377484 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:51.377450 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:52.705063 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:52.705033 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:52.762689 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:52.762661 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:55.272154 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:55.272104 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:55.924104 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:55.924070 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:56.025984 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:56.025954 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:00:59.788559 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:00:59.788527 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:09.277698 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:09.277666 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:09.378364 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:09.378338 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:11.586145 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.586109 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88"] Jul 06 12:01:11.586540 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.586497 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d4d566c-a768-49ef-a8a7-c62f22563fb7" containerName="main" Jul 06 12:01:11.586540 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.586509 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d4d566c-a768-49ef-a8a7-c62f22563fb7" containerName="main" Jul 06 12:01:11.586613 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.586582 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d4d566c-a768-49ef-a8a7-c62f22563fb7" containerName="main" Jul 06 12:01:11.589759 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.589740 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.591787 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.591768 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6862d7\"/\"kube-root-ca.crt\"" Jul 06 12:01:11.592355 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.592330 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6862d7\"/\"test-model-6862d7-kserve-self-signed-certs\"" Jul 06 12:01:11.592453 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.592337 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6862d7\"/\"openshift-service-ca.crt\"" Jul 06 12:01:11.592453 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.592372 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6862d7\"/\"default-dockercfg-n9chn\"" Jul 06 12:01:11.599691 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.599671 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88"] Jul 06 12:01:11.618044 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.618021 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:11.638478 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.638454 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-model-cache\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.638544 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.638497 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-dshm\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.638586 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.638560 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-tmp-dir\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.638620 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.638594 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c07081-a78a-47f9-93c9-c9a54582a241-tls-certs\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.638661 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.638625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-home\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.638697 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.638658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7j95\" (UniqueName: \"kubernetes.io/projected/52c07081-a78a-47f9-93c9-c9a54582a241-kube-api-access-z7j95\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.739758 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.739735 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-model-cache\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.739869 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.739777 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-dshm\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.739869 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.739817 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-tmp-dir\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.739869 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.739846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c07081-a78a-47f9-93c9-c9a54582a241-tls-certs\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.740054 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.739875 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-home\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.740054 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.739918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7j95\" (UniqueName: \"kubernetes.io/projected/52c07081-a78a-47f9-93c9-c9a54582a241-kube-api-access-z7j95\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.740220 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.740193 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-model-cache\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.740379 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.740360 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-home\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.740432 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.740402 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-tmp-dir\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.742091 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.742064 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-dshm\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.742220 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.742203 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c07081-a78a-47f9-93c9-c9a54582a241-tls-certs\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.748022 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.747999 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7j95\" (UniqueName: \"kubernetes.io/projected/52c07081-a78a-47f9-93c9-c9a54582a241-kube-api-access-z7j95\") pod \"test-model-6862d7-kserve-5d76588bbc-5pd88\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:11.899566 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:11.899511 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:12.033050 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:12.033015 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88"] Jul 06 12:01:12.035312 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:01:12.035284 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52c07081_a78a_47f9_93c9_c9a54582a241.slice/crio-1c1a72ab1ac0389dee3fc4d6c09087576ac161fe1bfff06d16b07bb4644faa57 WatchSource:0}: Error finding container 1c1a72ab1ac0389dee3fc4d6c09087576ac161fe1bfff06d16b07bb4644faa57: Status 404 returned error can't find the container with id 1c1a72ab1ac0389dee3fc4d6c09087576ac161fe1bfff06d16b07bb4644faa57 Jul 06 12:01:12.343777 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:12.343746 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" event={"ID":"52c07081-a78a-47f9-93c9-c9a54582a241","Type":"ContainerStarted","Data":"1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f"} Jul 06 12:01:12.343924 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:12.343787 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" event={"ID":"52c07081-a78a-47f9-93c9-c9a54582a241","Type":"ContainerStarted","Data":"1c1a72ab1ac0389dee3fc4d6c09087576ac161fe1bfff06d16b07bb4644faa57"} Jul 06 12:01:12.358700 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:12.358653 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" podStartSLOduration=1.358633278 podStartE2EDuration="1.358633278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:01:12.357866262 +0000 UTC m=+1941.889605587" watchObservedRunningTime="2026-07-06 12:01:12.358633278 +0000 UTC m=+1941.890372605" Jul 06 12:01:16.575087 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:16.575056 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:16.675851 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:16.675807 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:21.900738 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:21.900703 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:21.900738 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:21.900748 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:21.913051 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:21.913032 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:22.403517 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:22.403492 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:01:43.995612 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:43.995579 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j"] Jul 06 12:01:43.999288 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:43.999261 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.001792 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.001760 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f369e4\"/\"test-model-f369e4-kserve-self-signed-certs\"" Jul 06 12:01:44.001960 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.001802 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f369e4\"/\"openshift-service-ca.crt\"" Jul 06 12:01:44.001960 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.001867 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f369e4\"/\"default-dockercfg-tnrsq\"" Jul 06 12:01:44.001960 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.001886 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f369e4\"/\"kube-root-ca.crt\"" Jul 06 12:01:44.008484 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.008455 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j"] Jul 06 12:01:44.026792 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.026763 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:01:44.087745 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.087712 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-home\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.087896 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.087756 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-model-cache\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.087896 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.087799 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-tmp-dir\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.088030 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.087941 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-dshm\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.088030 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.087978 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66d2152-bb7d-479c-83e9-90944034959a-tls-certs\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.088106 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.088030 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntbsk\" (UniqueName: \"kubernetes.io/projected/b66d2152-bb7d-479c-83e9-90944034959a-kube-api-access-ntbsk\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.188881 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.188854 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-home\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.188881 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.188882 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-model-cache\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189032 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.188907 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-tmp-dir\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189032 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.188968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-dshm\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189032 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.188997 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66d2152-bb7d-479c-83e9-90944034959a-tls-certs\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189207 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.189039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntbsk\" (UniqueName: \"kubernetes.io/projected/b66d2152-bb7d-479c-83e9-90944034959a-kube-api-access-ntbsk\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189406 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.189379 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-home\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189406 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.189402 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-tmp-dir\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.189545 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.189385 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-model-cache\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.191443 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.191425 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-dshm\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.191642 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.191622 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66d2152-bb7d-479c-83e9-90944034959a-tls-certs\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.203008 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.202986 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntbsk\" (UniqueName: \"kubernetes.io/projected/b66d2152-bb7d-479c-83e9-90944034959a-kube-api-access-ntbsk\") pod \"test-model-f369e4-kserve-5dcf87899-b6b8j\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.309702 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.309650 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:44.444354 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.444328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j"] Jul 06 12:01:44.502004 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:44.501981 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" event={"ID":"b66d2152-bb7d-479c-83e9-90944034959a","Type":"ContainerStarted","Data":"94269b088c9f96f208d4d3f15a86e4ea8f30c6b24f9237933ebe262594c60b87"} Jul 06 12:01:45.509244 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:45.509207 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" event={"ID":"b66d2152-bb7d-479c-83e9-90944034959a","Type":"ContainerStarted","Data":"4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a"} Jul 06 12:01:45.525505 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:45.525460 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" podStartSLOduration=2.525445254 podStartE2EDuration="2.525445254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:01:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:01:45.52432503 +0000 UTC m=+1975.056064356" watchObservedRunningTime="2026-07-06 12:01:45.525445254 +0000 UTC m=+1975.057184580" Jul 06 12:01:54.310111 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:54.310077 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:54.310111 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:54.310115 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:54.322577 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:54.322550 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:01:54.565101 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:01:54.565033 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:02:18.724598 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:18.724560 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88"] Jul 06 12:02:18.741386 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:18.741355 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:02:19.748726 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:19.748697 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j"] Jul 06 12:02:19.768310 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:19.768272 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:02:20.459147 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:20.459108 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:02:20.558968 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:20.558930 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:02:20.659749 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:20.659704 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:02:25.390866 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:25.390826 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:02:33.748551 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:33.748386 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" podUID="52c07081-a78a-47f9-93c9-c9a54582a241" containerName="main" containerID="cri-o://1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f" gracePeriod=45 Jul 06 12:02:34.006227 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.006175 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:02:34.086712 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.086687 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7j95\" (UniqueName: \"kubernetes.io/projected/52c07081-a78a-47f9-93c9-c9a54582a241-kube-api-access-z7j95\") pod \"52c07081-a78a-47f9-93c9-c9a54582a241\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " Jul 06 12:02:34.086837 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.086725 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c07081-a78a-47f9-93c9-c9a54582a241-tls-certs\") pod \"52c07081-a78a-47f9-93c9-c9a54582a241\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " Jul 06 12:02:34.086837 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.086800 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-tmp-dir\") pod \"52c07081-a78a-47f9-93c9-c9a54582a241\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " Jul 06 12:02:34.086837 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.086824 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-model-cache\") pod \"52c07081-a78a-47f9-93c9-c9a54582a241\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " Jul 06 12:02:34.087011 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.086846 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-dshm\") pod \"52c07081-a78a-47f9-93c9-c9a54582a241\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " Jul 06 12:02:34.087011 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.086871 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-home\") pod \"52c07081-a78a-47f9-93c9-c9a54582a241\" (UID: \"52c07081-a78a-47f9-93c9-c9a54582a241\") " Jul 06 12:02:34.087129 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.087106 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-model-cache" (OuterVolumeSpecName: "model-cache") pod "52c07081-a78a-47f9-93c9-c9a54582a241" (UID: "52c07081-a78a-47f9-93c9-c9a54582a241"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:34.087129 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.087114 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52c07081-a78a-47f9-93c9-c9a54582a241" (UID: "52c07081-a78a-47f9-93c9-c9a54582a241"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:34.087268 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.087121 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-home" (OuterVolumeSpecName: "home") pod "52c07081-a78a-47f9-93c9-c9a54582a241" (UID: "52c07081-a78a-47f9-93c9-c9a54582a241"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:34.087268 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.087217 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:34.087268 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.087233 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:34.087268 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.087243 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:34.088974 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.088953 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52c07081-a78a-47f9-93c9-c9a54582a241-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52c07081-a78a-47f9-93c9-c9a54582a241" (UID: "52c07081-a78a-47f9-93c9-c9a54582a241"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:02:34.089056 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.089003 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c07081-a78a-47f9-93c9-c9a54582a241-kube-api-access-z7j95" (OuterVolumeSpecName: "kube-api-access-z7j95") pod "52c07081-a78a-47f9-93c9-c9a54582a241" (UID: "52c07081-a78a-47f9-93c9-c9a54582a241"). InnerVolumeSpecName "kube-api-access-z7j95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:02:34.089109 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.089056 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-dshm" (OuterVolumeSpecName: "dshm") pod "52c07081-a78a-47f9-93c9-c9a54582a241" (UID: "52c07081-a78a-47f9-93c9-c9a54582a241"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:34.188132 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.188109 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7j95\" (UniqueName: \"kubernetes.io/projected/52c07081-a78a-47f9-93c9-c9a54582a241-kube-api-access-z7j95\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:34.188132 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.188132 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52c07081-a78a-47f9-93c9-c9a54582a241-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:34.188277 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.188140 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52c07081-a78a-47f9-93c9-c9a54582a241-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:34.757883 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.757854 2585 generic.go:356] "Generic (PLEG): container finished" podID="52c07081-a78a-47f9-93c9-c9a54582a241" containerID="1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f" exitCode=0 Jul 06 12:02:34.758231 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.757896 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" Jul 06 12:02:34.758231 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.757929 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" event={"ID":"52c07081-a78a-47f9-93c9-c9a54582a241","Type":"ContainerDied","Data":"1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f"} Jul 06 12:02:34.758231 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.757963 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88" event={"ID":"52c07081-a78a-47f9-93c9-c9a54582a241","Type":"ContainerDied","Data":"1c1a72ab1ac0389dee3fc4d6c09087576ac161fe1bfff06d16b07bb4644faa57"} Jul 06 12:02:34.758231 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.757980 2585 scope.go:117] "RemoveContainer" containerID="1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f" Jul 06 12:02:34.767291 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.767271 2585 scope.go:117] "RemoveContainer" containerID="1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f" Jul 06 12:02:34.767535 ip-10-0-143-173 kubenswrapper[2585]: E0706 12:02:34.767517 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f\": container with ID starting with 1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f not found: ID does not exist" containerID="1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f" Jul 06 12:02:34.767580 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.767544 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f"} err="failed to get container status \"1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f\": rpc error: code = NotFound desc = could not find container \"1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f\": container with ID starting with 1347bf2fd5e15d83696e96bdafb3a5be38e734e343b481369b9491e06438676f not found: ID does not exist" Jul 06 12:02:34.768814 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.768771 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" podUID="b66d2152-bb7d-479c-83e9-90944034959a" containerName="main" containerID="cri-o://4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a" gracePeriod=45 Jul 06 12:02:34.779640 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.779618 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88"] Jul 06 12:02:34.783919 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:34.783900 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6862d7/test-model-6862d7-kserve-5d76588bbc-5pd88"] Jul 06 12:02:35.000635 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.000603 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c07081-a78a-47f9-93c9-c9a54582a241" path="/var/lib/kubelet/pods/52c07081-a78a-47f9-93c9-c9a54582a241/volumes" Jul 06 12:02:35.021674 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.021623 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:02:35.096707 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096685 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-home\") pod \"b66d2152-bb7d-479c-83e9-90944034959a\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " Jul 06 12:02:35.096822 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096728 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-dshm\") pod \"b66d2152-bb7d-479c-83e9-90944034959a\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " Jul 06 12:02:35.096822 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096771 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66d2152-bb7d-479c-83e9-90944034959a-tls-certs\") pod \"b66d2152-bb7d-479c-83e9-90944034959a\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " Jul 06 12:02:35.096822 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096805 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-model-cache\") pod \"b66d2152-bb7d-479c-83e9-90944034959a\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " Jul 06 12:02:35.096991 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096824 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntbsk\" (UniqueName: \"kubernetes.io/projected/b66d2152-bb7d-479c-83e9-90944034959a-kube-api-access-ntbsk\") pod \"b66d2152-bb7d-479c-83e9-90944034959a\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " Jul 06 12:02:35.096991 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096883 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-tmp-dir\") pod \"b66d2152-bb7d-479c-83e9-90944034959a\" (UID: \"b66d2152-bb7d-479c-83e9-90944034959a\") " Jul 06 12:02:35.096991 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.096910 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-home" (OuterVolumeSpecName: "home") pod "b66d2152-bb7d-479c-83e9-90944034959a" (UID: "b66d2152-bb7d-479c-83e9-90944034959a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:35.097188 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.097096 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-model-cache" (OuterVolumeSpecName: "model-cache") pod "b66d2152-bb7d-479c-83e9-90944034959a" (UID: "b66d2152-bb7d-479c-83e9-90944034959a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:35.097250 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.097206 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b66d2152-bb7d-479c-83e9-90944034959a" (UID: "b66d2152-bb7d-479c-83e9-90944034959a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:35.097250 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.097233 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:35.097350 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.097255 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:35.099006 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.098983 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b66d2152-bb7d-479c-83e9-90944034959a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b66d2152-bb7d-479c-83e9-90944034959a" (UID: "b66d2152-bb7d-479c-83e9-90944034959a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:02:35.099303 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.099286 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b66d2152-bb7d-479c-83e9-90944034959a-kube-api-access-ntbsk" (OuterVolumeSpecName: "kube-api-access-ntbsk") pod "b66d2152-bb7d-479c-83e9-90944034959a" (UID: "b66d2152-bb7d-479c-83e9-90944034959a"). InnerVolumeSpecName "kube-api-access-ntbsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:02:35.099367 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.099289 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-dshm" (OuterVolumeSpecName: "dshm") pod "b66d2152-bb7d-479c-83e9-90944034959a" (UID: "b66d2152-bb7d-479c-83e9-90944034959a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:02:35.197938 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.197917 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:35.197938 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.197937 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66d2152-bb7d-479c-83e9-90944034959a-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:35.198058 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.197948 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntbsk\" (UniqueName: \"kubernetes.io/projected/b66d2152-bb7d-479c-83e9-90944034959a-kube-api-access-ntbsk\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:35.198058 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.197956 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b66d2152-bb7d-479c-83e9-90944034959a-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:02:35.764905 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.764875 2585 generic.go:356] "Generic (PLEG): container finished" podID="b66d2152-bb7d-479c-83e9-90944034959a" containerID="4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a" exitCode=0 Jul 06 12:02:35.765393 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.764918 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" Jul 06 12:02:35.765393 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.764958 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" event={"ID":"b66d2152-bb7d-479c-83e9-90944034959a","Type":"ContainerDied","Data":"4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a"} Jul 06 12:02:35.765393 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.764996 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j" event={"ID":"b66d2152-bb7d-479c-83e9-90944034959a","Type":"ContainerDied","Data":"94269b088c9f96f208d4d3f15a86e4ea8f30c6b24f9237933ebe262594c60b87"} Jul 06 12:02:35.765393 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.765013 2585 scope.go:117] "RemoveContainer" containerID="4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a" Jul 06 12:02:35.773808 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.773793 2585 scope.go:117] "RemoveContainer" containerID="4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a" Jul 06 12:02:35.774043 ip-10-0-143-173 kubenswrapper[2585]: E0706 12:02:35.774016 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a\": container with ID starting with 4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a not found: ID does not exist" containerID="4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a" Jul 06 12:02:35.774090 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.774041 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a"} err="failed to get container status \"4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a\": rpc error: code = NotFound desc = could not find container \"4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a\": container with ID starting with 4e01e10911c9191f551e2a16ac5e66ce52e4cd54eaf5c8adaa737234037a738a not found: ID does not exist" Jul 06 12:02:35.796182 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.796135 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j"] Jul 06 12:02:35.805343 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:35.805325 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f369e4/test-model-f369e4-kserve-5dcf87899-b6b8j"] Jul 06 12:02:36.999522 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:02:36.999481 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b66d2152-bb7d-479c-83e9-90944034959a" path="/var/lib/kubelet/pods/b66d2152-bb7d-479c-83e9-90944034959a/volumes" Jul 06 12:03:02.665853 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:02.665816 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:02.950303 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:02.950224 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:03.199733 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:03.199701 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:03.469678 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:03.469643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:03.992887 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:03.992854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:04.093400 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:04.093372 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:04.327177 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:04.327135 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:10.028069 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:10.028031 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:10.132784 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:10.132753 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:10.336065 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:10.335989 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:10.610326 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:10.610258 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:11.020853 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:11.020823 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:11.374190 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:11.374097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:11.617361 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:11.617330 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:14.765571 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:14.765490 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:14.839056 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:14.839025 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:14.906490 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:14.906461 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:15.005256 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:15.005214 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:15.433659 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:15.433631 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:15.531986 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:15.531960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:19.230928 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:19.230897 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:19.696766 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:19.696735 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:20.004270 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:20.004248 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:20.099619 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:20.099596 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:20.468387 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:20.468315 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:22.980811 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:22.980781 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:23.060461 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:23.060434 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:23.122332 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:23.122306 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:23.663763 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:23.663737 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:23.780975 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:23.780952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:23.917885 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:23.917827 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:27.469649 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:27.469617 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:35.485699 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:35.485662 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:35.833023 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:35.832992 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:35.933407 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:35.933378 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:36.033907 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:36.033882 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:36.184823 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:36.184747 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:39.016997 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:39.016965 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:39.098668 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:39.098641 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:41.861978 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:41.861940 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:41.962304 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:41.962280 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:42.079309 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:42.079285 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:42.308388 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:42.308358 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:42.499604 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:42.499580 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:42.975267 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:42.975197 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:43.075351 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:43.075324 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:43.238908 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:43.238842 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:48.919887 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:48.919854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:49.019940 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:49.019910 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:49.171551 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:49.171474 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:49.400638 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:49.400610 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:49.722414 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:49.722385 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:50.024316 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:50.024292 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:50.347842 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:50.347780 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:52.946018 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:52.945986 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:53.008232 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:53.008203 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:53.069759 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:53.069724 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:53.357483 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:53.357446 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:53.606120 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:53.606094 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:53.707837 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:53.707770 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:57.460828 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:57.460798 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:57.758100 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:57.758073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:58.192200 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:58.192117 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:03:58.964611 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:03:58.964580 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:01.164358 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:01.164324 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:01.241045 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:01.241014 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:01.303937 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:01.303910 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:01.426978 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:01.426903 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:01.858117 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:01.858083 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:01.956813 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:01.956787 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:05.675255 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:05.675220 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:15.467433 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:15.467399 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:15.570059 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:15.570029 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:22.595102 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:22.595073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:22.696067 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:22.696042 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:44.173198 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173155 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t"] Jul 06 12:04:44.173607 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173520 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c07081-a78a-47f9-93c9-c9a54582a241" containerName="main" Jul 06 12:04:44.173607 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173531 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c07081-a78a-47f9-93c9-c9a54582a241" containerName="main" Jul 06 12:04:44.173607 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173540 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b66d2152-bb7d-479c-83e9-90944034959a" containerName="main" Jul 06 12:04:44.173607 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173546 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b66d2152-bb7d-479c-83e9-90944034959a" containerName="main" Jul 06 12:04:44.173738 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173619 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b66d2152-bb7d-479c-83e9-90944034959a" containerName="main" Jul 06 12:04:44.173738 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.173629 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c07081-a78a-47f9-93c9-c9a54582a241" containerName="main" Jul 06 12:04:44.176762 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.176735 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.178650 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.178629 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"default-dockercfg-nvbpw\"" Jul 06 12:04:44.179136 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.179111 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"kube-root-ca.crt\"" Jul 06 12:04:44.179369 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.179353 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"auth-test-model-8b7fb6-kserve-self-signed-certs\"" Jul 06 12:04:44.179438 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.179368 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"openshift-service-ca.crt\"" Jul 06 12:04:44.186330 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.186308 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t"] Jul 06 12:04:44.197196 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.197178 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-model-cache\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.197281 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.197205 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-dshm\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.197281 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.197239 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdlm4\" (UniqueName: \"kubernetes.io/projected/e1331df1-dcbb-448d-807d-59576753b9ec-kube-api-access-bdlm4\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.197357 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.197314 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1331df1-dcbb-448d-807d-59576753b9ec-tls-certs\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.197357 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.197339 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-home\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.197422 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.197358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-tmp-dir\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.200889 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.200872 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:04:44.297858 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.297834 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1331df1-dcbb-448d-807d-59576753b9ec-tls-certs\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.297952 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.297864 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-home\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.297952 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.297884 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-tmp-dir\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.297952 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.297915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-model-cache\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.297952 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.297941 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-dshm\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.298135 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.297992 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdlm4\" (UniqueName: \"kubernetes.io/projected/e1331df1-dcbb-448d-807d-59576753b9ec-kube-api-access-bdlm4\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.298302 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.298281 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-home\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.298382 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.298336 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-model-cache\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.298433 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.298413 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-tmp-dir\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.300267 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.300234 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-dshm\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.300503 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.300487 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1331df1-dcbb-448d-807d-59576753b9ec-tls-certs\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.305686 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.305667 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdlm4\" (UniqueName: \"kubernetes.io/projected/e1331df1-dcbb-448d-807d-59576753b9ec-kube-api-access-bdlm4\") pod \"auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.486509 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.486456 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:44.615951 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:44.615927 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t"] Jul 06 12:04:44.618474 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:04:44.618447 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1331df1_dcbb_448d_807d_59576753b9ec.slice/crio-c08259a3dea260c8503021aa631fc48a10f25233cce0e6e1ade5d05c72a20a59 WatchSource:0}: Error finding container c08259a3dea260c8503021aa631fc48a10f25233cce0e6e1ade5d05c72a20a59: Status 404 returned error can't find the container with id c08259a3dea260c8503021aa631fc48a10f25233cce0e6e1ade5d05c72a20a59 Jul 06 12:04:45.392860 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:45.392821 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" event={"ID":"e1331df1-dcbb-448d-807d-59576753b9ec","Type":"ContainerStarted","Data":"cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f"} Jul 06 12:04:45.393255 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:45.392867 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" event={"ID":"e1331df1-dcbb-448d-807d-59576753b9ec","Type":"ContainerStarted","Data":"c08259a3dea260c8503021aa631fc48a10f25233cce0e6e1ade5d05c72a20a59"} Jul 06 12:04:45.412577 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:45.412533 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" podStartSLOduration=1.412518032 podStartE2EDuration="1.412518032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:04:45.411387111 +0000 UTC m=+2154.943126438" watchObservedRunningTime="2026-07-06 12:04:45.412518032 +0000 UTC m=+2154.944257358" Jul 06 12:04:54.486969 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:54.486937 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:54.487439 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:54.486984 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:54.499464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:54.499441 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:04:55.451499 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:04:55.451468 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:05:01.788694 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.788658 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k"] Jul 06 12:05:01.792504 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.792485 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.794476 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.794457 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"kube-root-ca.crt\"" Jul 06 12:05:01.795043 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.795023 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"default-dockercfg-w8f9w\"" Jul 06 12:05:01.795137 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.795096 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"openshift-service-ca.crt\"" Jul 06 12:05:01.795236 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.795223 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"auth-test-model-cf8c6a-kserve-self-signed-certs\"" Jul 06 12:05:01.799605 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.799583 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k"] Jul 06 12:05:01.835538 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.835517 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-dshm\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.835647 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.835554 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-tmp-dir\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.835647 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.835592 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9aa5cda-d020-4a11-8092-af51d5f3574d-tls-certs\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.835732 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.835666 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-home\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.835732 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.835694 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8wn6\" (UniqueName: \"kubernetes.io/projected/d9aa5cda-d020-4a11-8092-af51d5f3574d-kube-api-access-v8wn6\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.835732 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.835715 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-model-cache\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936203 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936180 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-dshm\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936304 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-tmp-dir\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936304 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936232 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9aa5cda-d020-4a11-8092-af51d5f3574d-tls-certs\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936304 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936253 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-home\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936304 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936275 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8wn6\" (UniqueName: \"kubernetes.io/projected/d9aa5cda-d020-4a11-8092-af51d5f3574d-kube-api-access-v8wn6\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936492 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-model-cache\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936669 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936644 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-tmp-dir\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936741 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936669 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-home\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.936817 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.936791 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-model-cache\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.938500 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.938477 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-dshm\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.938632 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.938613 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9aa5cda-d020-4a11-8092-af51d5f3574d-tls-certs\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:01.943110 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:01.943092 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8wn6\" (UniqueName: \"kubernetes.io/projected/d9aa5cda-d020-4a11-8092-af51d5f3574d-kube-api-access-v8wn6\") pod \"auth-test-model-cf8c6a-kserve-559596b7f9-fm95k\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:02.105395 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:02.105331 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:02.183273 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:02.183246 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:05:02.235613 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:02.235579 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k"] Jul 06 12:05:02.239029 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:05:02.239002 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9aa5cda_d020_4a11_8092_af51d5f3574d.slice/crio-92ed00d06020a163369682331f4053936e814f4059b1f87206bf4d1fec8f16ab WatchSource:0}: Error finding container 92ed00d06020a163369682331f4053936e814f4059b1f87206bf4d1fec8f16ab: Status 404 returned error can't find the container with id 92ed00d06020a163369682331f4053936e814f4059b1f87206bf4d1fec8f16ab Jul 06 12:05:02.472760 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:02.472683 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" event={"ID":"d9aa5cda-d020-4a11-8092-af51d5f3574d","Type":"ContainerStarted","Data":"5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a"} Jul 06 12:05:02.472760 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:02.472724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" event={"ID":"d9aa5cda-d020-4a11-8092-af51d5f3574d","Type":"ContainerStarted","Data":"92ed00d06020a163369682331f4053936e814f4059b1f87206bf4d1fec8f16ab"} Jul 06 12:05:02.490627 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:02.490584 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" podStartSLOduration=1.490569482 podStartE2EDuration="1.490569482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:05:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:05:02.488513019 +0000 UTC m=+2172.020252343" watchObservedRunningTime="2026-07-06 12:05:02.490569482 +0000 UTC m=+2172.022308808" Jul 06 12:05:11.068873 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:11.068829 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:05:12.106086 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:12.106056 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:12.106597 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:12.106099 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:12.118482 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:12.118457 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:05:12.531346 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:05:12.531323 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:06:14.516861 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:14.516771 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:06:17.883457 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:17.883422 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:06:20.236085 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:20.236044 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:06:20.352355 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:20.352328 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:06:58.707728 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:58.707692 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t"] Jul 06 12:06:58.730486 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:58.730458 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:06:59.693138 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:59.693106 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k"] Jul 06 12:06:59.702302 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:06:59.702276 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:07:08.732142 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:08.732103 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:07:13.727513 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:13.727471 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" podUID="e1331df1-dcbb-448d-807d-59576753b9ec" containerName="main" containerID="cri-o://cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f" gracePeriod=45 Jul 06 12:07:13.976431 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:13.976407 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:07:14.055898 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.055873 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-tmp-dir\") pod \"e1331df1-dcbb-448d-807d-59576753b9ec\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " Jul 06 12:07:14.056018 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.055928 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1331df1-dcbb-448d-807d-59576753b9ec-tls-certs\") pod \"e1331df1-dcbb-448d-807d-59576753b9ec\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " Jul 06 12:07:14.056018 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.055950 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-home\") pod \"e1331df1-dcbb-448d-807d-59576753b9ec\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " Jul 06 12:07:14.056018 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.055973 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdlm4\" (UniqueName: \"kubernetes.io/projected/e1331df1-dcbb-448d-807d-59576753b9ec-kube-api-access-bdlm4\") pod \"e1331df1-dcbb-448d-807d-59576753b9ec\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " Jul 06 12:07:14.056018 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056011 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-dshm\") pod \"e1331df1-dcbb-448d-807d-59576753b9ec\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " Jul 06 12:07:14.056228 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056030 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-model-cache\") pod \"e1331df1-dcbb-448d-807d-59576753b9ec\" (UID: \"e1331df1-dcbb-448d-807d-59576753b9ec\") " Jul 06 12:07:14.056228 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056137 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e1331df1-dcbb-448d-807d-59576753b9ec" (UID: "e1331df1-dcbb-448d-807d-59576753b9ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:14.056337 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056235 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-home" (OuterVolumeSpecName: "home") pod "e1331df1-dcbb-448d-807d-59576753b9ec" (UID: "e1331df1-dcbb-448d-807d-59576753b9ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:14.056473 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056448 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:14.056637 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056476 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:14.056637 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.056548 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "e1331df1-dcbb-448d-807d-59576753b9ec" (UID: "e1331df1-dcbb-448d-807d-59576753b9ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:14.058218 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.058197 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-dshm" (OuterVolumeSpecName: "dshm") pod "e1331df1-dcbb-448d-807d-59576753b9ec" (UID: "e1331df1-dcbb-448d-807d-59576753b9ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:14.058218 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.058210 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1331df1-dcbb-448d-807d-59576753b9ec-kube-api-access-bdlm4" (OuterVolumeSpecName: "kube-api-access-bdlm4") pod "e1331df1-dcbb-448d-807d-59576753b9ec" (UID: "e1331df1-dcbb-448d-807d-59576753b9ec"). InnerVolumeSpecName "kube-api-access-bdlm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:07:14.058340 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.058289 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1331df1-dcbb-448d-807d-59576753b9ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e1331df1-dcbb-448d-807d-59576753b9ec" (UID: "e1331df1-dcbb-448d-807d-59576753b9ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:07:14.113180 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.113129 2585 generic.go:356] "Generic (PLEG): container finished" podID="e1331df1-dcbb-448d-807d-59576753b9ec" containerID="cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f" exitCode=0 Jul 06 12:07:14.113284 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.113202 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" Jul 06 12:07:14.113284 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.113197 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" event={"ID":"e1331df1-dcbb-448d-807d-59576753b9ec","Type":"ContainerDied","Data":"cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f"} Jul 06 12:07:14.113366 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.113306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t" event={"ID":"e1331df1-dcbb-448d-807d-59576753b9ec","Type":"ContainerDied","Data":"c08259a3dea260c8503021aa631fc48a10f25233cce0e6e1ade5d05c72a20a59"} Jul 06 12:07:14.113366 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.113328 2585 scope.go:117] "RemoveContainer" containerID="cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f" Jul 06 12:07:14.122810 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.122795 2585 scope.go:117] "RemoveContainer" containerID="cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f" Jul 06 12:07:14.123093 ip-10-0-143-173 kubenswrapper[2585]: E0706 12:07:14.123075 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f\": container with ID starting with cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f not found: ID does not exist" containerID="cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f" Jul 06 12:07:14.123140 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.123100 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f"} err="failed to get container status \"cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f\": rpc error: code = NotFound desc = could not find container \"cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f\": container with ID starting with cdd44dad01f3a4fef8bc427ca9e4432ee8050bf9835fffd4232b435544cc349f not found: ID does not exist" Jul 06 12:07:14.133764 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.133740 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t"] Jul 06 12:07:14.137587 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.137568 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/auth-test-model-8b7fb6-kserve-5b56649bc5-hrp8t"] Jul 06 12:07:14.157502 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.157482 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:14.157502 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.157501 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1331df1-dcbb-448d-807d-59576753b9ec-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:14.157611 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.157511 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1331df1-dcbb-448d-807d-59576753b9ec-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:14.157611 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.157520 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdlm4\" (UniqueName: \"kubernetes.io/projected/e1331df1-dcbb-448d-807d-59576753b9ec-kube-api-access-bdlm4\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:14.712704 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.712656 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" podUID="d9aa5cda-d020-4a11-8092-af51d5f3574d" containerName="main" containerID="cri-o://5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a" gracePeriod=45 Jul 06 12:07:14.962534 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:14.962502 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:07:15.005000 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.004972 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1331df1-dcbb-448d-807d-59576753b9ec" path="/var/lib/kubelet/pods/e1331df1-dcbb-448d-807d-59576753b9ec/volumes" Jul 06 12:07:15.063503 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063481 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-dshm\") pod \"d9aa5cda-d020-4a11-8092-af51d5f3574d\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " Jul 06 12:07:15.063609 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063518 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8wn6\" (UniqueName: \"kubernetes.io/projected/d9aa5cda-d020-4a11-8092-af51d5f3574d-kube-api-access-v8wn6\") pod \"d9aa5cda-d020-4a11-8092-af51d5f3574d\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " Jul 06 12:07:15.063609 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063571 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9aa5cda-d020-4a11-8092-af51d5f3574d-tls-certs\") pod \"d9aa5cda-d020-4a11-8092-af51d5f3574d\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " Jul 06 12:07:15.063609 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063586 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-tmp-dir\") pod \"d9aa5cda-d020-4a11-8092-af51d5f3574d\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " Jul 06 12:07:15.063721 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063617 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-home\") pod \"d9aa5cda-d020-4a11-8092-af51d5f3574d\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " Jul 06 12:07:15.063721 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063644 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-model-cache\") pod \"d9aa5cda-d020-4a11-8092-af51d5f3574d\" (UID: \"d9aa5cda-d020-4a11-8092-af51d5f3574d\") " Jul 06 12:07:15.063893 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.063844 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d9aa5cda-d020-4a11-8092-af51d5f3574d" (UID: "d9aa5cda-d020-4a11-8092-af51d5f3574d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:15.064234 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.064027 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d9aa5cda-d020-4a11-8092-af51d5f3574d" (UID: "d9aa5cda-d020-4a11-8092-af51d5f3574d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:15.064351 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.064237 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-home" (OuterVolumeSpecName: "home") pod "d9aa5cda-d020-4a11-8092-af51d5f3574d" (UID: "d9aa5cda-d020-4a11-8092-af51d5f3574d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:15.065678 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.065658 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-dshm" (OuterVolumeSpecName: "dshm") pod "d9aa5cda-d020-4a11-8092-af51d5f3574d" (UID: "d9aa5cda-d020-4a11-8092-af51d5f3574d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:07:15.066050 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.066034 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9aa5cda-d020-4a11-8092-af51d5f3574d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d9aa5cda-d020-4a11-8092-af51d5f3574d" (UID: "d9aa5cda-d020-4a11-8092-af51d5f3574d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:07:15.066128 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.066057 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9aa5cda-d020-4a11-8092-af51d5f3574d-kube-api-access-v8wn6" (OuterVolumeSpecName: "kube-api-access-v8wn6") pod "d9aa5cda-d020-4a11-8092-af51d5f3574d" (UID: "d9aa5cda-d020-4a11-8092-af51d5f3574d"). InnerVolumeSpecName "kube-api-access-v8wn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:07:15.120656 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.120629 2585 generic.go:356] "Generic (PLEG): container finished" podID="d9aa5cda-d020-4a11-8092-af51d5f3574d" containerID="5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a" exitCode=0 Jul 06 12:07:15.120763 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.120674 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" Jul 06 12:07:15.120763 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.120713 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" event={"ID":"d9aa5cda-d020-4a11-8092-af51d5f3574d","Type":"ContainerDied","Data":"5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a"} Jul 06 12:07:15.120880 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.120764 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k" event={"ID":"d9aa5cda-d020-4a11-8092-af51d5f3574d","Type":"ContainerDied","Data":"92ed00d06020a163369682331f4053936e814f4059b1f87206bf4d1fec8f16ab"} Jul 06 12:07:15.120880 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.120791 2585 scope.go:117] "RemoveContainer" containerID="5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a" Jul 06 12:07:15.129367 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.129353 2585 scope.go:117] "RemoveContainer" containerID="5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a" Jul 06 12:07:15.129624 ip-10-0-143-173 kubenswrapper[2585]: E0706 12:07:15.129606 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a\": container with ID starting with 5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a not found: ID does not exist" containerID="5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a" Jul 06 12:07:15.129665 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.129636 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a"} err="failed to get container status \"5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a\": rpc error: code = NotFound desc = could not find container \"5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a\": container with ID starting with 5649d90f0b4efbc091d8581ae90351eb152c0d4e127e0b3338597ec6c384fe6a not found: ID does not exist" Jul 06 12:07:15.140102 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.140082 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k"] Jul 06 12:07:15.144411 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.144390 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/auth-test-model-cf8c6a-kserve-559596b7f9-fm95k"] Jul 06 12:07:15.165315 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.165297 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9aa5cda-d020-4a11-8092-af51d5f3574d-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:15.165315 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.165314 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:15.165428 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.165322 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:15.165428 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.165330 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:15.165428 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.165338 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9aa5cda-d020-4a11-8092-af51d5f3574d-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:15.165428 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:15.165346 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8wn6\" (UniqueName: \"kubernetes.io/projected/d9aa5cda-d020-4a11-8092-af51d5f3574d-kube-api-access-v8wn6\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:07:16.999604 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:16.999573 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9aa5cda-d020-4a11-8092-af51d5f3574d" path="/var/lib/kubelet/pods/d9aa5cda-d020-4a11-8092-af51d5f3574d/volumes" Jul 06 12:07:18.502065 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502029 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w"] Jul 06 12:07:18.502496 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502429 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9aa5cda-d020-4a11-8092-af51d5f3574d" containerName="main" Jul 06 12:07:18.502496 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502441 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9aa5cda-d020-4a11-8092-af51d5f3574d" containerName="main" Jul 06 12:07:18.502496 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502452 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1331df1-dcbb-448d-807d-59576753b9ec" containerName="main" Jul 06 12:07:18.502496 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502458 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1331df1-dcbb-448d-807d-59576753b9ec" containerName="main" Jul 06 12:07:18.502628 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502519 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9aa5cda-d020-4a11-8092-af51d5f3574d" containerName="main" Jul 06 12:07:18.502628 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.502531 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1331df1-dcbb-448d-807d-59576753b9ec" containerName="main" Jul 06 12:07:18.506810 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.506788 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.508838 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.508818 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"openshift-service-ca.crt\"" Jul 06 12:07:18.508965 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.508844 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"kube-root-ca.crt\"" Jul 06 12:07:18.508965 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.508851 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"default-dockercfg-nvbpw\"" Jul 06 12:07:18.508965 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.508855 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8b7fb6\"/\"sub-test-model-8b7fb6-kserve-self-signed-certs\"" Jul 06 12:07:18.513581 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.513558 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w"] Jul 06 12:07:18.583186 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.583148 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:07:18.590398 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.590383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tmp-dir\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.590464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.590419 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-dshm\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.590464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.590442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-home\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.590464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.590457 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwcl6\" (UniqueName: \"kubernetes.io/projected/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-kube-api-access-qwcl6\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.590595 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.590518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-model-cache\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.590595 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.590552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tls-certs\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.691841 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.691816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-home\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.691958 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.691844 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwcl6\" (UniqueName: \"kubernetes.io/projected/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-kube-api-access-qwcl6\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.691958 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.691866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-model-cache\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.691958 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.691887 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tls-certs\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.691958 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.691940 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tmp-dir\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.692216 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.691972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-dshm\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.692301 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.692284 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-home\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.692351 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.692335 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-model-cache\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.692417 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.692402 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tmp-dir\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.694214 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.694193 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-dshm\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.694456 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.694440 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tls-certs\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.700036 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.700020 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwcl6\" (UniqueName: \"kubernetes.io/projected/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-kube-api-access-qwcl6\") pod \"sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.817335 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.817290 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:18.944942 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:18.944918 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w"] Jul 06 12:07:18.947701 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:07:18.947674 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4b9ab82_ab8a_4a8d_9fa5_2b5061eef94e.slice/crio-249ec816835e5e09b67ec91947c42fad8f4ed93d9b7fdf485535f23309a72d4c WatchSource:0}: Error finding container 249ec816835e5e09b67ec91947c42fad8f4ed93d9b7fdf485535f23309a72d4c: Status 404 returned error can't find the container with id 249ec816835e5e09b67ec91947c42fad8f4ed93d9b7fdf485535f23309a72d4c Jul 06 12:07:19.145666 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:19.145587 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" event={"ID":"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e","Type":"ContainerStarted","Data":"8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8"} Jul 06 12:07:19.145666 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:19.145631 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" event={"ID":"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e","Type":"ContainerStarted","Data":"249ec816835e5e09b67ec91947c42fad8f4ed93d9b7fdf485535f23309a72d4c"} Jul 06 12:07:19.165644 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:19.165597 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" podStartSLOduration=1.165581287 podStartE2EDuration="1.165581287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:07:19.163443137 +0000 UTC m=+2308.695182476" watchObservedRunningTime="2026-07-06 12:07:19.165581287 +0000 UTC m=+2308.697320612" Jul 06 12:07:28.818354 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:28.818309 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:28.818354 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:28.818367 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:28.831041 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:28.831015 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:29.203690 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:29.203616 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:07:40.205228 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.205195 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2"] Jul 06 12:07:40.208650 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.208629 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.212120 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.211454 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"kube-root-ca.crt\"" Jul 06 12:07:40.212120 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.211962 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"openshift-service-ca.crt\"" Jul 06 12:07:40.216476 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.212641 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"sub-test-model-cf8c6a-kserve-self-signed-certs\"" Jul 06 12:07:40.216476 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.213028 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"default-dockercfg-w8f9w\"" Jul 06 12:07:40.222181 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.222139 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2"] Jul 06 12:07:40.298988 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.298959 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:07:40.351229 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.351203 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-tmp-dir\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.351229 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.351229 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-dshm\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.351375 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.351261 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4rp5\" (UniqueName: \"kubernetes.io/projected/77cff106-ec6d-4789-b68e-8f53cbdba73e-kube-api-access-r4rp5\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.351375 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.351285 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-home\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.351375 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.351352 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-model-cache\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.351489 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.351385 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cff106-ec6d-4789-b68e-8f53cbdba73e-tls-certs\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.451939 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.451897 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-tmp-dir\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.451939 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.451943 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-dshm\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452115 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.451972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4rp5\" (UniqueName: \"kubernetes.io/projected/77cff106-ec6d-4789-b68e-8f53cbdba73e-kube-api-access-r4rp5\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452115 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.451990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-home\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452115 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.452024 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-model-cache\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452115 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.452051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cff106-ec6d-4789-b68e-8f53cbdba73e-tls-certs\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452425 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.452405 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-tmp-dir\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452493 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.452471 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-model-cache\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.452538 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.452509 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-home\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.454126 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.454107 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-dshm\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.454612 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.454592 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cff106-ec6d-4789-b68e-8f53cbdba73e-tls-certs\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.460374 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.460330 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4rp5\" (UniqueName: \"kubernetes.io/projected/77cff106-ec6d-4789-b68e-8f53cbdba73e-kube-api-access-r4rp5\") pod \"sub-test-model-cf8c6a-kserve-59d5787485-9xdh2\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.526458 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.526435 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:40.653424 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:40.653391 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2"] Jul 06 12:07:40.655331 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:07:40.655301 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77cff106_ec6d_4789_b68e_8f53cbdba73e.slice/crio-40b7b94f17a5a98721c5cee47fdddec64932c469a60f5d155b5307e49185bd0c WatchSource:0}: Error finding container 40b7b94f17a5a98721c5cee47fdddec64932c469a60f5d155b5307e49185bd0c: Status 404 returned error can't find the container with id 40b7b94f17a5a98721c5cee47fdddec64932c469a60f5d155b5307e49185bd0c Jul 06 12:07:41.256355 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:41.256321 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" event={"ID":"77cff106-ec6d-4789-b68e-8f53cbdba73e","Type":"ContainerStarted","Data":"c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e"} Jul 06 12:07:41.256754 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:41.256365 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" event={"ID":"77cff106-ec6d-4789-b68e-8f53cbdba73e","Type":"ContainerStarted","Data":"40b7b94f17a5a98721c5cee47fdddec64932c469a60f5d155b5307e49185bd0c"} Jul 06 12:07:41.274720 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:41.274657 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" podStartSLOduration=1.2746353240000001 podStartE2EDuration="1.274635324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:07:41.274423343 +0000 UTC m=+2330.806162700" watchObservedRunningTime="2026-07-06 12:07:41.274635324 +0000 UTC m=+2330.806374650" Jul 06 12:07:50.527124 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:50.527034 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:50.527124 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:50.527081 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:50.539697 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:50.539673 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:07:51.316707 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:07:51.316682 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:08:21.174844 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:21.174812 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w"] Jul 06 12:08:21.193637 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:21.193609 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:08:22.188552 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:22.188517 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2"] Jul 06 12:08:22.233852 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:22.233810 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:08:23.840786 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:23.840725 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:08:25.662614 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:25.662574 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:08:36.194599 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.194555 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" podUID="e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" containerName="main" containerID="cri-o://8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8" gracePeriod=45 Jul 06 12:08:36.445871 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.445819 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:08:36.487811 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.487779 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwcl6\" (UniqueName: \"kubernetes.io/projected/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-kube-api-access-qwcl6\") pod \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " Jul 06 12:08:36.487941 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.487844 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-dshm\") pod \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " Jul 06 12:08:36.487941 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.487896 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tls-certs\") pod \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " Jul 06 12:08:36.488058 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.487951 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tmp-dir\") pod \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " Jul 06 12:08:36.488058 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.487980 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-home\") pod \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " Jul 06 12:08:36.488058 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.488042 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-model-cache\") pod \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\" (UID: \"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e\") " Jul 06 12:08:36.488384 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.488292 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" (UID: "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:36.488384 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.488314 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-home" (OuterVolumeSpecName: "home") pod "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" (UID: "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:36.488545 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.488393 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:36.488545 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.488512 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" (UID: "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:36.489981 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.489955 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" (UID: "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:08:36.489981 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.489956 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-kube-api-access-qwcl6" (OuterVolumeSpecName: "kube-api-access-qwcl6") pod "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" (UID: "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e"). InnerVolumeSpecName "kube-api-access-qwcl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:08:36.490263 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.490250 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-dshm" (OuterVolumeSpecName: "dshm") pod "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" (UID: "e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:36.545567 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.545543 2585 generic.go:356] "Generic (PLEG): container finished" podID="e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" containerID="8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8" exitCode=0 Jul 06 12:08:36.545660 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.545582 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" Jul 06 12:08:36.545660 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.545605 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" event={"ID":"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e","Type":"ContainerDied","Data":"8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8"} Jul 06 12:08:36.545660 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.545636 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w" event={"ID":"e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e","Type":"ContainerDied","Data":"249ec816835e5e09b67ec91947c42fad8f4ed93d9b7fdf485535f23309a72d4c"} Jul 06 12:08:36.545660 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.545657 2585 scope.go:117] "RemoveContainer" containerID="8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8" Jul 06 12:08:36.555293 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.555278 2585 scope.go:117] "RemoveContainer" containerID="8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8" Jul 06 12:08:36.555543 ip-10-0-143-173 kubenswrapper[2585]: E0706 12:08:36.555526 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8\": container with ID starting with 8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8 not found: ID does not exist" containerID="8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8" Jul 06 12:08:36.555596 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.555551 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8"} err="failed to get container status \"8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8\": rpc error: code = NotFound desc = could not find container \"8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8\": container with ID starting with 8f6935f769713722d1f9822c4c2c3507e50e88a5960d50ddc8275b997ea53ec8 not found: ID does not exist" Jul 06 12:08:36.568400 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.568378 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w"] Jul 06 12:08:36.574564 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.574547 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8b7fb6/sub-test-model-8b7fb6-kserve-f86c7ccfc-qmz7w"] Jul 06 12:08:36.589455 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.589432 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwcl6\" (UniqueName: \"kubernetes.io/projected/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-kube-api-access-qwcl6\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:36.589455 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.589451 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:36.589577 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.589462 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:36.589577 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.589471 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:36.589577 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.589478 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:36.998880 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:36.998851 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" path="/var/lib/kubelet/pods/e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e/volumes" Jul 06 12:08:37.208799 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.208767 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" podUID="77cff106-ec6d-4789-b68e-8f53cbdba73e" containerName="main" containerID="cri-o://c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e" gracePeriod=45 Jul 06 12:08:37.454776 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.454751 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:08:37.495111 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495084 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4rp5\" (UniqueName: \"kubernetes.io/projected/77cff106-ec6d-4789-b68e-8f53cbdba73e-kube-api-access-r4rp5\") pod \"77cff106-ec6d-4789-b68e-8f53cbdba73e\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " Jul 06 12:08:37.495261 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495142 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-dshm\") pod \"77cff106-ec6d-4789-b68e-8f53cbdba73e\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " Jul 06 12:08:37.495261 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495199 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-home\") pod \"77cff106-ec6d-4789-b68e-8f53cbdba73e\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " Jul 06 12:08:37.495261 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495243 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-tmp-dir\") pod \"77cff106-ec6d-4789-b68e-8f53cbdba73e\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " Jul 06 12:08:37.495417 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495268 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cff106-ec6d-4789-b68e-8f53cbdba73e-tls-certs\") pod \"77cff106-ec6d-4789-b68e-8f53cbdba73e\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " Jul 06 12:08:37.495417 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495291 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-model-cache\") pod \"77cff106-ec6d-4789-b68e-8f53cbdba73e\" (UID: \"77cff106-ec6d-4789-b68e-8f53cbdba73e\") " Jul 06 12:08:37.495782 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495758 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77cff106-ec6d-4789-b68e-8f53cbdba73e" (UID: "77cff106-ec6d-4789-b68e-8f53cbdba73e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:37.496042 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.495999 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-model-cache" (OuterVolumeSpecName: "model-cache") pod "77cff106-ec6d-4789-b68e-8f53cbdba73e" (UID: "77cff106-ec6d-4789-b68e-8f53cbdba73e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:37.496195 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.496140 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-home" (OuterVolumeSpecName: "home") pod "77cff106-ec6d-4789-b68e-8f53cbdba73e" (UID: "77cff106-ec6d-4789-b68e-8f53cbdba73e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:37.497645 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.497615 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77cff106-ec6d-4789-b68e-8f53cbdba73e-kube-api-access-r4rp5" (OuterVolumeSpecName: "kube-api-access-r4rp5") pod "77cff106-ec6d-4789-b68e-8f53cbdba73e" (UID: "77cff106-ec6d-4789-b68e-8f53cbdba73e"). InnerVolumeSpecName "kube-api-access-r4rp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:08:37.497968 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.497936 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-dshm" (OuterVolumeSpecName: "dshm") pod "77cff106-ec6d-4789-b68e-8f53cbdba73e" (UID: "77cff106-ec6d-4789-b68e-8f53cbdba73e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:08:37.498273 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.498250 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77cff106-ec6d-4789-b68e-8f53cbdba73e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77cff106-ec6d-4789-b68e-8f53cbdba73e" (UID: "77cff106-ec6d-4789-b68e-8f53cbdba73e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:08:37.551719 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.551686 2585 generic.go:356] "Generic (PLEG): container finished" podID="77cff106-ec6d-4789-b68e-8f53cbdba73e" containerID="c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e" exitCode=0 Jul 06 12:08:37.551794 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.551746 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" Jul 06 12:08:37.551794 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.551767 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" event={"ID":"77cff106-ec6d-4789-b68e-8f53cbdba73e","Type":"ContainerDied","Data":"c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e"} Jul 06 12:08:37.551877 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.551808 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2" event={"ID":"77cff106-ec6d-4789-b68e-8f53cbdba73e","Type":"ContainerDied","Data":"40b7b94f17a5a98721c5cee47fdddec64932c469a60f5d155b5307e49185bd0c"} Jul 06 12:08:37.551877 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.551826 2585 scope.go:117] "RemoveContainer" containerID="c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e" Jul 06 12:08:37.561282 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.561267 2585 scope.go:117] "RemoveContainer" containerID="c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e" Jul 06 12:08:37.561542 ip-10-0-143-173 kubenswrapper[2585]: E0706 12:08:37.561522 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e\": container with ID starting with c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e not found: ID does not exist" containerID="c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e" Jul 06 12:08:37.561595 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.561550 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e"} err="failed to get container status \"c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e\": rpc error: code = NotFound desc = could not find container \"c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e\": container with ID starting with c6482ef23cf13e66dfc565d6df71a282411f7b2c49113da83fa2811f996e3a5e not found: ID does not exist" Jul 06 12:08:37.572836 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.572813 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2"] Jul 06 12:08:37.574618 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.574577 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/sub-test-model-cf8c6a-kserve-59d5787485-9xdh2"] Jul 06 12:08:37.596330 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.596309 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:37.596330 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.596330 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:37.596461 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.596343 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:37.596461 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.596354 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77cff106-ec6d-4789-b68e-8f53cbdba73e-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:37.596461 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.596366 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77cff106-ec6d-4789-b68e-8f53cbdba73e-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:37.596461 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:37.596378 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4rp5\" (UniqueName: \"kubernetes.io/projected/77cff106-ec6d-4789-b68e-8f53cbdba73e-kube-api-access-r4rp5\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:08:38.999923 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:38.999883 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77cff106-ec6d-4789-b68e-8f53cbdba73e" path="/var/lib/kubelet/pods/77cff106-ec6d-4789-b68e-8f53cbdba73e/volumes" Jul 06 12:08:39.716813 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:39.716783 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:08:39.795496 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:08:39.795468 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:09:09.190804 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.190756 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m"] Jul 06 12:09:09.191390 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.191176 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" containerName="main" Jul 06 12:09:09.191390 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.191189 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" containerName="main" Jul 06 12:09:09.191390 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.191209 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77cff106-ec6d-4789-b68e-8f53cbdba73e" containerName="main" Jul 06 12:09:09.191390 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.191216 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="77cff106-ec6d-4789-b68e-8f53cbdba73e" containerName="main" Jul 06 12:09:09.191390 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.191309 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="77cff106-ec6d-4789-b68e-8f53cbdba73e" containerName="main" Jul 06 12:09:09.191390 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.191320 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4b9ab82-ab8a-4a8d-9fa5-2b5061eef94e" containerName="main" Jul 06 12:09:09.194518 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.194496 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.196534 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.196513 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"kube-root-ca.crt\"" Jul 06 12:09:09.196999 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.196976 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"default-dockercfg-w8f9w\"" Jul 06 12:09:09.197111 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.197078 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"openshift-service-ca.crt\"" Jul 06 12:09:09.197203 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.197128 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cf8c6a\"/\"rate-test-model-cf8c6a-kserve-self-signed-certs\"" Jul 06 12:09:09.204244 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.204220 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m"] Jul 06 12:09:09.223385 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.223358 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:09:09.339808 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.339782 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-model-cache\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.339916 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.339814 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tls-certs\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.339916 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.339846 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tmp-dir\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.340041 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.339941 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-home\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.340041 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.339970 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-dshm\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.340041 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.339993 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8p26\" (UniqueName: \"kubernetes.io/projected/bc33d979-1a8d-45ef-8774-5beb10f0ce41-kube-api-access-v8p26\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440639 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440613 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tls-certs\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440737 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tmp-dir\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440798 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440776 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-home\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440877 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440822 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-dshm\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440877 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440857 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8p26\" (UniqueName: \"kubernetes.io/projected/bc33d979-1a8d-45ef-8774-5beb10f0ce41-kube-api-access-v8p26\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440995 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440923 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-model-cache\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.440995 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.440956 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tmp-dir\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.441184 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.441139 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-home\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.441323 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.441305 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-model-cache\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.442968 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.442942 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-dshm\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.443304 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.443286 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tls-certs\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.448330 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.448308 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8p26\" (UniqueName: \"kubernetes.io/projected/bc33d979-1a8d-45ef-8774-5beb10f0ce41-kube-api-access-v8p26\") pod \"rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.509531 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.509508 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:09.654399 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.654373 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m"] Jul 06 12:09:09.656401 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:09:09.656376 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc33d979_1a8d_45ef_8774_5beb10f0ce41.slice/crio-031fb685b97ff5882951b7e6a64846304f8fd989593c02f0d808ffc2caff9796 WatchSource:0}: Error finding container 031fb685b97ff5882951b7e6a64846304f8fd989593c02f0d808ffc2caff9796: Status 404 returned error can't find the container with id 031fb685b97ff5882951b7e6a64846304f8fd989593c02f0d808ffc2caff9796 Jul 06 12:09:09.707941 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:09.707907 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" event={"ID":"bc33d979-1a8d-45ef-8774-5beb10f0ce41","Type":"ContainerStarted","Data":"031fb685b97ff5882951b7e6a64846304f8fd989593c02f0d808ffc2caff9796"} Jul 06 12:09:10.714629 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:10.714588 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" event={"ID":"bc33d979-1a8d-45ef-8774-5beb10f0ce41","Type":"ContainerStarted","Data":"e0e3cd670fb82d1d97676ee97c10d8eb4b6b426493511c23f12ea89c8d94df9a"} Jul 06 12:09:10.733635 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:10.733581 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" podStartSLOduration=1.733565907 podStartE2EDuration="1.733565907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:09:10.732206014 +0000 UTC m=+2420.263945360" watchObservedRunningTime="2026-07-06 12:09:10.733565907 +0000 UTC m=+2420.265305235" Jul 06 12:09:16.280365 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:16.280289 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:09:16.380889 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:16.380860 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:09:19.510515 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:19.510479 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:19.511009 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:19.510530 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:19.522846 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:19.522818 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:19.768838 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:19.768765 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:09:23.295686 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:23.295651 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:09:24.021095 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:09:24.021060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:14.962716 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:14.962678 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:15.878868 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:15.878836 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m"] Jul 06 12:10:15.898486 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:15.898462 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:17.082577 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:17.082546 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:17.439054 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:17.438959 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:17.548085 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:17.548057 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:17.640382 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:17.640359 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:17.785417 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:17.785392 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:22.515773 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:22.515721 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:30.899305 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:30.899266 2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" podUID="bc33d979-1a8d-45ef-8774-5beb10f0ce41" containerName="main" containerID="cri-o://e0e3cd670fb82d1d97676ee97c10d8eb4b6b426493511c23f12ea89c8d94df9a" gracePeriod=45 Jul 06 12:10:31.118938 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.118904 2585 generic.go:356] "Generic (PLEG): container finished" podID="bc33d979-1a8d-45ef-8774-5beb10f0ce41" containerID="e0e3cd670fb82d1d97676ee97c10d8eb4b6b426493511c23f12ea89c8d94df9a" exitCode=0 Jul 06 12:10:31.119077 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.118973 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" event={"ID":"bc33d979-1a8d-45ef-8774-5beb10f0ce41","Type":"ContainerDied","Data":"e0e3cd670fb82d1d97676ee97c10d8eb4b6b426493511c23f12ea89c8d94df9a"} Jul 06 12:10:31.143386 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.143363 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:10:31.276620 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276596 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-model-cache\") pod \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " Jul 06 12:10:31.276752 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276624 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tls-certs\") pod \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " Jul 06 12:10:31.276752 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276659 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8p26\" (UniqueName: \"kubernetes.io/projected/bc33d979-1a8d-45ef-8774-5beb10f0ce41-kube-api-access-v8p26\") pod \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " Jul 06 12:10:31.276752 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276683 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tmp-dir\") pod \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " Jul 06 12:10:31.276752 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276714 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-dshm\") pod \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " Jul 06 12:10:31.276752 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276753 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-home\") pod \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\" (UID: \"bc33d979-1a8d-45ef-8774-5beb10f0ce41\") " Jul 06 12:10:31.277013 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276900 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc33d979-1a8d-45ef-8774-5beb10f0ce41" (UID: "bc33d979-1a8d-45ef-8774-5beb10f0ce41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:10:31.277013 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.276957 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc33d979-1a8d-45ef-8774-5beb10f0ce41" (UID: "bc33d979-1a8d-45ef-8774-5beb10f0ce41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:10:31.277137 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.277043 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-home" (OuterVolumeSpecName: "home") pod "bc33d979-1a8d-45ef-8774-5beb10f0ce41" (UID: "bc33d979-1a8d-45ef-8774-5beb10f0ce41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:10:31.277137 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.277052 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tmp-dir\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:10:31.277137 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.277083 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-model-cache\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:10:31.278755 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.278718 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-dshm" (OuterVolumeSpecName: "dshm") pod "bc33d979-1a8d-45ef-8774-5beb10f0ce41" (UID: "bc33d979-1a8d-45ef-8774-5beb10f0ce41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:10:31.278844 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.278771 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc33d979-1a8d-45ef-8774-5beb10f0ce41" (UID: "bc33d979-1a8d-45ef-8774-5beb10f0ce41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:10:31.278844 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.278775 2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc33d979-1a8d-45ef-8774-5beb10f0ce41-kube-api-access-v8p26" (OuterVolumeSpecName: "kube-api-access-v8p26") pod "bc33d979-1a8d-45ef-8774-5beb10f0ce41" (UID: "bc33d979-1a8d-45ef-8774-5beb10f0ce41"). InnerVolumeSpecName "kube-api-access-v8p26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:10:31.378060 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.378037 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-dshm\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:10:31.378060 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.378055 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc33d979-1a8d-45ef-8774-5beb10f0ce41-home\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:10:31.378060 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.378064 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc33d979-1a8d-45ef-8774-5beb10f0ce41-tls-certs\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:10:31.378253 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:31.378073 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8p26\" (UniqueName: \"kubernetes.io/projected/bc33d979-1a8d-45ef-8774-5beb10f0ce41-kube-api-access-v8p26\") on node \"ip-10-0-143-173.ec2.internal\" DevicePath \"\"" Jul 06 12:10:32.125079 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:32.125046 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" Jul 06 12:10:32.125079 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:32.125069 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m" event={"ID":"bc33d979-1a8d-45ef-8774-5beb10f0ce41","Type":"ContainerDied","Data":"031fb685b97ff5882951b7e6a64846304f8fd989593c02f0d808ffc2caff9796"} Jul 06 12:10:32.125633 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:32.125123 2585 scope.go:117] "RemoveContainer" containerID="e0e3cd670fb82d1d97676ee97c10d8eb4b6b426493511c23f12ea89c8d94df9a" Jul 06 12:10:32.145909 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:32.145887 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m"] Jul 06 12:10:32.149222 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:32.149190 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cf8c6a/rate-test-model-cf8c6a-kserve-65b4b59479-hsd9m"] Jul 06 12:10:32.999267 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:32.999238 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc33d979-1a8d-45ef-8774-5beb10f0ce41" path="/var/lib/kubelet/pods/bc33d979-1a8d-45ef-8774-5beb10f0ce41/volumes" Jul 06 12:10:59.408091 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:59.408052 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:59.686930 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:59.686839 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:10:59.906373 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:10:59.906349 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:00.087264 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:00.087241 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:00.510375 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:00.510343 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:00.611655 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:00.611626 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:00.728562 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:00.728528 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:06.425268 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:06.425231 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:06.485855 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:06.485825 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:06.697070 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:06.696994 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:06.920445 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:06.920416 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:07.333139 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:07.333110 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8586b869f9-rdz4k_e2a0c7b2-1f82-4b97-894e-c4e28cecf125/authorino/0.log" Jul 06 12:11:13.135152 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:13.135121 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb7946974-t5qpg_9393dbc6-a81e-402d-a3fe-5a2a05c97db8/copy-manifests/0.log" Jul 06 12:11:13.142467 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:13.142447 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb7946974-t5qpg_9393dbc6-a81e-402d-a3fe-5a2a05c97db8/manager/0.log" Jul 06 12:11:14.582488 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:14.582461 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8586b869f9-rdz4k_e2a0c7b2-1f82-4b97-894e-c4e28cecf125/authorino/0.log" Jul 06 12:11:14.710489 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:14.710461 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pb92d_f189420f-28d2-4950-bb3f-aad22c04a93c/manager/0.log" Jul 06 12:11:16.146310 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:16.146285 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c57c544d-c2hml_1bfd5cde-5fb9-40a0-b857-33193bee4a9d/kube-auth-proxy/0.log" Jul 06 12:11:16.383548 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:16.383515 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5gd74_78b95f8b-e3e1-43ab-889f-7e17463c6a50/payload-pre-processing/0.log" Jul 06 12:11:17.205037 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:17.205009 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-q26hx_189c68ff-cdd4-499a-a334-6e8c27c34cbe/main/0.log" Jul 06 12:11:17.550355 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:17.550323 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-6kclt_7a0d7385-5906-4d37-8cc1-134728f45c16/main/0.log" Jul 06 12:11:22.120400 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.120356 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qzjtz/must-gather-tzfhr"] Jul 06 12:11:22.120931 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.120911 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc33d979-1a8d-45ef-8774-5beb10f0ce41" containerName="main" Jul 06 12:11:22.121013 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.120934 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc33d979-1a8d-45ef-8774-5beb10f0ce41" containerName="main" Jul 06 12:11:22.121082 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.121068 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc33d979-1a8d-45ef-8774-5beb10f0ce41" containerName="main" Jul 06 12:11:22.124759 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.124731 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.127206 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.127091 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzjtz\"/\"openshift-service-ca.crt\"" Jul 06 12:11:22.127206 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.127092 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzjtz\"/\"kube-root-ca.crt\"" Jul 06 12:11:22.127579 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.127561 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qzjtz\"/\"default-dockercfg-jkctc\"" Jul 06 12:11:22.130621 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.130599 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzjtz/must-gather-tzfhr"] Jul 06 12:11:22.132154 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.132125 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:22.270216 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.270193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tkw7\" (UniqueName: \"kubernetes.io/projected/0df0dc7c-d295-4e7f-b563-dd6bb969bc46-kube-api-access-9tkw7\") pod \"must-gather-tzfhr\" (UID: \"0df0dc7c-d295-4e7f-b563-dd6bb969bc46\") " pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.270345 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.270229 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0df0dc7c-d295-4e7f-b563-dd6bb969bc46-must-gather-output\") pod \"must-gather-tzfhr\" (UID: \"0df0dc7c-d295-4e7f-b563-dd6bb969bc46\") " pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.371104 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.371040 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tkw7\" (UniqueName: \"kubernetes.io/projected/0df0dc7c-d295-4e7f-b563-dd6bb969bc46-kube-api-access-9tkw7\") pod \"must-gather-tzfhr\" (UID: \"0df0dc7c-d295-4e7f-b563-dd6bb969bc46\") " pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.371104 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.371079 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0df0dc7c-d295-4e7f-b563-dd6bb969bc46-must-gather-output\") pod \"must-gather-tzfhr\" (UID: \"0df0dc7c-d295-4e7f-b563-dd6bb969bc46\") " pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.371392 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.371374 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0df0dc7c-d295-4e7f-b563-dd6bb969bc46-must-gather-output\") pod \"must-gather-tzfhr\" (UID: \"0df0dc7c-d295-4e7f-b563-dd6bb969bc46\") " pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.378689 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.378658 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tkw7\" (UniqueName: \"kubernetes.io/projected/0df0dc7c-d295-4e7f-b563-dd6bb969bc46-kube-api-access-9tkw7\") pod \"must-gather-tzfhr\" (UID: \"0df0dc7c-d295-4e7f-b563-dd6bb969bc46\") " pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.434966 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.434943 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzjtz/must-gather-tzfhr" Jul 06 12:11:22.561800 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.561680 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzjtz/must-gather-tzfhr"] Jul 06 12:11:22.564437 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:11:22.564411 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0df0dc7c_d295_4e7f_b563_dd6bb969bc46.slice/crio-6653bb8fb820bd8aa173aefb6ff42799e307791dae01f30b2828972fe61169bf WatchSource:0}: Error finding container 6653bb8fb820bd8aa173aefb6ff42799e307791dae01f30b2828972fe61169bf: Status 404 returned error can't find the container with id 6653bb8fb820bd8aa173aefb6ff42799e307791dae01f30b2828972fe61169bf Jul 06 12:11:22.566325 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:22.566309 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 12:11:23.364660 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:23.364633 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjtz/must-gather-tzfhr" event={"ID":"0df0dc7c-d295-4e7f-b563-dd6bb969bc46","Type":"ContainerStarted","Data":"6653bb8fb820bd8aa173aefb6ff42799e307791dae01f30b2828972fe61169bf"} Jul 06 12:11:23.708506 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:23.708440 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" limit=1024 Jul 06 12:11:24.373258 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:24.373223 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjtz/must-gather-tzfhr" event={"ID":"0df0dc7c-d295-4e7f-b563-dd6bb969bc46","Type":"ContainerStarted","Data":"7f38be812c7c66c4172c78f9fdcc9db6b670fdca4b2750e4d60f3077d37d18e1"} Jul 06 12:11:24.373671 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:24.373275 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjtz/must-gather-tzfhr" event={"ID":"0df0dc7c-d295-4e7f-b563-dd6bb969bc46","Type":"ContainerStarted","Data":"988c03264f517404e4c8755cf3fbca9080bfa97afcf721c51a5e40694431de48"} Jul 06 12:11:24.390914 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:24.390869 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qzjtz/must-gather-tzfhr" podStartSLOduration=1.6499301800000001 podStartE2EDuration="2.390852403s" totalImagesPullingTime="740.922223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:11:24.389213264 +0000 UTC m=+2553.920952591" watchObservedRunningTime="2026-07-06 12:11:24.390852403 +0000 UTC m=+2553.922591729" Jul 06 12:11:24.928521 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:24.928489 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q8q7d_5a6ad56c-e987-47b6-a64c-acf6705a3228/global-pull-secret-syncer/0.log" Jul 06 12:11:25.032890 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:25.032861 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fsvg9_b5b8fa1d-0de0-4339-bfc7-1e940522963c/konnectivity-agent/0.log" Jul 06 12:11:25.187138 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:25.187042 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-173.ec2.internal_088e17fe59b4ec12c1114c83bdf80edd/haproxy/0.log" Jul 06 12:11:30.254174 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:30.253009 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8586b869f9-rdz4k_e2a0c7b2-1f82-4b97-894e-c4e28cecf125/authorino/0.log" Jul 06 12:11:30.281398 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:30.281362 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pb92d_f189420f-28d2-4950-bb3f-aad22c04a93c/manager/0.log" Jul 06 12:11:32.350712 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.350681 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/alertmanager/0.log" Jul 06 12:11:32.379509 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.379434 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/config-reloader/0.log" Jul 06 12:11:32.404945 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.404912 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/kube-rbac-proxy-web/0.log" Jul 06 12:11:32.429930 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.429897 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/kube-rbac-proxy/0.log" Jul 06 12:11:32.452437 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.452403 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/kube-rbac-proxy-metric/0.log" Jul 06 12:11:32.478068 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.478015 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/prom-label-proxy/0.log" Jul 06 12:11:32.500614 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.500589 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_23b8c1ff-528e-48c3-a99b-d245b95612ca/init-config-reloader/0.log" Jul 06 12:11:32.578989 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.578948 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-hlxwv_5d3a800a-9d8f-4e0a-b917-d9186c24a12c/kube-state-metrics/0.log" Jul 06 12:11:32.599786 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.599759 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-hlxwv_5d3a800a-9d8f-4e0a-b917-d9186c24a12c/kube-rbac-proxy-main/0.log" Jul 06 12:11:32.621821 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.621754 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-hlxwv_5d3a800a-9d8f-4e0a-b917-d9186c24a12c/kube-rbac-proxy-self/0.log" Jul 06 12:11:32.652377 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.652353 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-57df598f5-v2n47_5432e6ef-38c8-48e6-b092-8562c8ae94a2/metrics-server/0.log" Jul 06 12:11:32.678284 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.678262 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-plzsp_c9eae277-ad42-4b0c-862d-0023de33d021/monitoring-plugin/0.log" Jul 06 12:11:32.795585 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.795554 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdpk_dcd49f24-f4df-48fc-848a-0fb607ee0f9b/node-exporter/0.log" Jul 06 12:11:32.816857 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.816832 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdpk_dcd49f24-f4df-48fc-848a-0fb607ee0f9b/kube-rbac-proxy/0.log" Jul 06 12:11:32.840136 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.840113 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bsdpk_dcd49f24-f4df-48fc-848a-0fb607ee0f9b/init-textfile/0.log" Jul 06 12:11:32.945742 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.945664 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-784f6_a3e9e6f5-6df7-4420-8180-585ed2f64161/kube-rbac-proxy-main/0.log" Jul 06 12:11:32.974116 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.974089 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-784f6_a3e9e6f5-6df7-4420-8180-585ed2f64161/kube-rbac-proxy-self/0.log" Jul 06 12:11:32.996657 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:32.996630 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-784f6_a3e9e6f5-6df7-4420-8180-585ed2f64161/openshift-state-metrics/0.log" Jul 06 12:11:33.316459 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.316429 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86c9b55c95-ndw48_0a0619f4-a51c-4c49-a6b3-d5e297a2f58b/telemeter-client/0.log" Jul 06 12:11:33.379459 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.379424 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86c9b55c95-ndw48_0a0619f4-a51c-4c49-a6b3-d5e297a2f58b/reload/0.log" Jul 06 12:11:33.409476 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.409445 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86c9b55c95-ndw48_0a0619f4-a51c-4c49-a6b3-d5e297a2f58b/kube-rbac-proxy/0.log" Jul 06 12:11:33.440378 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.440349 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5c478986-z444g_edfbdc71-00c7-40e7-840a-4958e4c55c5d/thanos-query/0.log" Jul 06 12:11:33.464315 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.464284 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5c478986-z444g_edfbdc71-00c7-40e7-840a-4958e4c55c5d/kube-rbac-proxy-web/0.log" Jul 06 12:11:33.488743 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.488717 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5c478986-z444g_edfbdc71-00c7-40e7-840a-4958e4c55c5d/kube-rbac-proxy/0.log" Jul 06 12:11:33.512373 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.512349 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5c478986-z444g_edfbdc71-00c7-40e7-840a-4958e4c55c5d/prom-label-proxy/0.log" Jul 06 12:11:33.548367 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.548342 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5c478986-z444g_edfbdc71-00c7-40e7-840a-4958e4c55c5d/kube-rbac-proxy-rules/0.log" Jul 06 12:11:33.571454 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:33.571380 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5c478986-z444g_edfbdc71-00c7-40e7-840a-4958e4c55c5d/kube-rbac-proxy-metrics/0.log" Jul 06 12:11:34.413280 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.413249 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9"] Jul 06 12:11:34.422440 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.422392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.426021 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.425990 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9"] Jul 06 12:11:34.517225 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.517190 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-podres\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.517437 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.517240 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-proc\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.517437 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.517271 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6lqn\" (UniqueName: \"kubernetes.io/projected/04f3434a-bcb8-4d15-b18d-d770d2404b3b-kube-api-access-n6lqn\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.517437 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.517376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-sys\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.517609 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.517463 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-lib-modules\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618293 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-lib-modules\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618345 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-podres\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618378 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-proc\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618400 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6lqn\" (UniqueName: \"kubernetes.io/projected/04f3434a-bcb8-4d15-b18d-d770d2404b3b-kube-api-access-n6lqn\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618464 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618462 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-sys\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618658 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618469 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-lib-modules\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618658 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618506 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-proc\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618658 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618529 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-sys\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.618658 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.618540 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/04f3434a-bcb8-4d15-b18d-d770d2404b3b-podres\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.625551 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.625526 2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6lqn\" (UniqueName: \"kubernetes.io/projected/04f3434a-bcb8-4d15-b18d-d770d2404b3b-kube-api-access-n6lqn\") pod \"perf-node-gather-daemonset-qfxq9\" (UID: \"04f3434a-bcb8-4d15-b18d-d770d2404b3b\") " pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.711185 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.711090 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-4tdp5_a48682a5-19b3-4f9b-ac7f-0e4ec2f89bda/networking-console-plugin/0.log" Jul 06 12:11:34.736744 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.736706 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:34.929180 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:34.929139 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9"] Jul 06 12:11:34.930834 ip-10-0-143-173 kubenswrapper[2585]: W0706 12:11:34.930807 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod04f3434a_bcb8_4d15_b18d_d770d2404b3b.slice/crio-0ed3527685cec0d525df5e0ef63852899b25761495d93d756bb7b47df9fb5777 WatchSource:0}: Error finding container 0ed3527685cec0d525df5e0ef63852899b25761495d93d756bb7b47df9fb5777: Status 404 returned error can't find the container with id 0ed3527685cec0d525df5e0ef63852899b25761495d93d756bb7b47df9fb5777 Jul 06 12:11:35.455820 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:35.455777 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" event={"ID":"04f3434a-bcb8-4d15-b18d-d770d2404b3b","Type":"ContainerStarted","Data":"9d376ad6f9c78f549c5421f5fa3ae6fcde6aba16f73daede0104e48bd5f61a21"} Jul 06 12:11:35.456357 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:35.455847 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" event={"ID":"04f3434a-bcb8-4d15-b18d-d770d2404b3b","Type":"ContainerStarted","Data":"0ed3527685cec0d525df5e0ef63852899b25761495d93d756bb7b47df9fb5777"} Jul 06 12:11:35.456662 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:35.456638 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:35.472347 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:35.472294 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" podStartSLOduration=1.47227332 podStartE2EDuration="1.47227332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:11:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:11:35.47081282 +0000 UTC m=+2565.002552147" watchObservedRunningTime="2026-07-06 12:11:35.47227332 +0000 UTC m=+2565.004012647" Jul 06 12:11:35.857263 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:35.857236 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6758ffdddb-fdq5v_5e71f2f4-c9e8-43ec-a476-1da17b419f8c/console/0.log" Jul 06 12:11:35.894808 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:35.894765 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-8bz8c_c6bec83a-0303-41cd-b5f1-0677571af709/download-server/0.log" Jul 06 12:11:37.500471 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:37.500435 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-467xf_bf26ed81-48f0-4641-8eb3-530e8699cc52/dns/0.log" Jul 06 12:11:37.522687 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:37.522660 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-467xf_bf26ed81-48f0-4641-8eb3-530e8699cc52/kube-rbac-proxy/0.log" Jul 06 12:11:37.646356 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:37.646329 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8lck9_50752636-200c-4d95-ad03-1c19fdc20645/dns-node-resolver/0.log" Jul 06 12:11:38.300666 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:38.300637 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5qngh_7bda73a4-7ae5-4c45-b4af-e4a49bb5d9a3/node-ca/0.log" Jul 06 12:11:39.517655 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:39.517626 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c57c544d-c2hml_1bfd5cde-5fb9-40a0-b857-33193bee4a9d/kube-auth-proxy/0.log" Jul 06 12:11:39.637763 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:39.637730 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5gd74_78b95f8b-e3e1-43ab-889f-7e17463c6a50/payload-pre-processing/0.log" Jul 06 12:11:40.458637 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:40.458610 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ss752_d44c25d8-3db3-400e-a94e-5b6a34b3c0f5/serve-healthcheck-canary/0.log" Jul 06 12:11:41.083751 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:41.083723 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xwdrh_8052bc97-28d9-466e-87fa-5435fbd1baa1/kube-rbac-proxy/0.log" Jul 06 12:11:41.106190 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:41.106151 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xwdrh_8052bc97-28d9-466e-87fa-5435fbd1baa1/exporter/0.log" Jul 06 12:11:41.128392 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:41.128357 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xwdrh_8052bc97-28d9-466e-87fa-5435fbd1baa1/extractor/0.log" Jul 06 12:11:42.484458 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:42.484430 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qzjtz/perf-node-gather-daemonset-qfxq9" Jul 06 12:11:44.000004 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:43.999965 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb7946974-t5qpg_9393dbc6-a81e-402d-a3fe-5a2a05c97db8/manager/0.log" Jul 06 12:11:44.023031 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:44.022994 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb7946974-t5qpg_9393dbc6-a81e-402d-a3fe-5a2a05c97db8/copy-manifests/0.log" Jul 06 12:11:44.504743 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:44.504693 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c57c544d-c2hml_1bfd5cde-5fb9-40a0-b857-33193bee4a9d/kube-auth-proxy/0.log" Jul 06 12:11:44.642521 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:44.642447 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5gd74_78b95f8b-e3e1-43ab-889f-7e17463c6a50/payload-pre-processing/0.log" Jul 06 12:11:45.525092 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:45.525014 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-758cc6698f-2ljdh_c285df41-15e0-4fa0-8e3b-7a0ab50adc31/manager/0.log" Jul 06 12:11:46.311996 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:46.311972 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c57c544d-c2hml_1bfd5cde-5fb9-40a0-b857-33193bee4a9d/kube-auth-proxy/0.log" Jul 06 12:11:46.413417 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:46.413388 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5gd74_78b95f8b-e3e1-43ab-889f-7e17463c6a50/payload-pre-processing/0.log" Jul 06 12:11:50.361193 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:50.361148 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-6whvp_c7fd9b79-a187-47ae-8cb6-ce5a46152687/migrator/0.log" Jul 06 12:11:50.388789 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:50.388760 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-6whvp_c7fd9b79-a187-47ae-8cb6-ce5a46152687/graceful-termination/0.log" Jul 06 12:11:51.992248 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:51.992222 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/kube-multus-additional-cni-plugins/0.log" Jul 06 12:11:52.014076 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.014053 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/egress-router-binary-copy/0.log" Jul 06 12:11:52.037985 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.037956 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/cni-plugins/0.log" Jul 06 12:11:52.064549 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.064523 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/bond-cni-plugin/0.log" Jul 06 12:11:52.091527 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.091499 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/routeoverride-cni/0.log" Jul 06 12:11:52.118230 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.118207 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/whereabouts-cni-bincopy/0.log" Jul 06 12:11:52.140691 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.140671 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8n4tf_49de6972-f0b8-4bc0-a83d-b837a63e3a6f/whereabouts-cni/0.log" Jul 06 12:11:52.422958 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.422934 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vwx7g_fc73ff93-6a36-47fb-b2f0-e85088d70097/kube-multus/0.log" Jul 06 12:11:52.443681 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.443661 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pfkrn_672720fa-209d-4354-8698-27024f8e88ec/network-metrics-daemon/0.log" Jul 06 12:11:52.464086 ip-10-0-143-173 kubenswrapper[2585]: I0706 12:11:52.464069 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pfkrn_672720fa-209d-4354-8698-27024f8e88ec/kube-rbac-proxy/0.log"