Oct 30 18:28:43.987424 ip-10-0-136-103 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 18:28:43.987433 ip-10-0-136-103 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 18:28:43.987439 ip-10-0-136-103 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 18:28:43.987661 ip-10-0-136-103 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 18:28:54.033658 ip-10-0-136-103 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 30 18:28:54.033673 ip-10-0-136-103 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b6958c44a3914f68a13bc624e3d89f67 -- Oct 30 18:30:57.655210 ip-10-0-136-103 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:30:58.021273 ip-10-0-136-103 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:58.021273 ip-10-0-136-103 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:30:58.021273 ip-10-0-136-103 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:58.021273 ip-10-0-136-103 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:58.026826 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026723 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.026826 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026821 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.026826 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026825 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.026826 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026828 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.026826 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026831 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.026826 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026834 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026837 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026839 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026842 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026845 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026847 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026850 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026853 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026855 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026858 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026862 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026865 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026867 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026870 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026873 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026875 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026878 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026880 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026883 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026886 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.027045 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026889 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026891 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026894 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026897 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026900 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026903 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026905 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026908 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026911 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026913 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026916 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026919 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026922 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026925 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026927 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026930 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026932 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026935 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026938 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026940 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.027518 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026943 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026946 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026948 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026970 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026973 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026977 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026980 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026984 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026986 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026989 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026992 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026994 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026997 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.026999 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027002 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027006 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027012 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027014 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027018 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.028015 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027021 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027024 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027027 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027030 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027032 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027035 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027039 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027041 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027044 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027046 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027049 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027052 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027055 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027058 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027061 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027063 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027066 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027069 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027072 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.028522 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027075 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027078 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027081 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027084 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027087 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027091 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027095 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027098 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027101 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027104 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027107 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027110 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027112 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027115 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027118 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027120 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027123 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027125 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027128 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.028974 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027132 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027134 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027137 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027139 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027913 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027920 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027923 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027926 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027929 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027931 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027938 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027970 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027974 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027978 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027985 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027989 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027992 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027995 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.027998 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.029402 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028000 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028003 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028006 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028008 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028011 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028013 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028016 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028019 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028021 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028024 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028027 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028029 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028032 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028035 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028038 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028041 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028043 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028046 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028048 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028051 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.029843 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028053 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028056 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028058 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028061 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028064 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028066 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028068 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028071 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028074 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028076 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028079 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028081 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028083 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028086 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028088 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028091 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028093 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028096 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028098 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.030324 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028100 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028103 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028105 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028108 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028111 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028114 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028116 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028119 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028122 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028124 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028127 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028129 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028131 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028134 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028136 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028138 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028141 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028145 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028149 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.030784 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028152 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028154 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028157 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028160 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028162 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028165 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028168 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028170 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028173 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028175 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028178 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028180 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028183 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028185 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028188 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028190 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028193 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028196 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028199 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028201 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.031253 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028205 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028207 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028210 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028212 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028214 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028217 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028219 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028222 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028224 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028227 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028229 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028232 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028234 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028707 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028712 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028716 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028718 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028721 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028724 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028727 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.031728 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028730 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028733 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028736 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028738 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028741 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028743 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028746 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028748 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028751 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028753 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028756 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028758 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028761 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028763 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028766 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028768 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028771 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028773 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028776 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028778 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.032234 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028781 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028783 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028785 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028788 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028793 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028795 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028797 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028800 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028803 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028805 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028808 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028810 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028814 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028818 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028821 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028824 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028826 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028829 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028831 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028834 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.032688 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028836 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028840 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028843 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028846 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028849 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028852 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028854 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028856 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028859 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028861 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028864 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028866 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028869 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028871 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028873 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028876 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028879 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028882 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028884 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.033142 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028887 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028889 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028892 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028895 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028897 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028901 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028904 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028907 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028909 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028912 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028914 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028917 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028919 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028922 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028925 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028927 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028929 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028932 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028934 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.033604 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028937 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028939 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028942 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028944 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028947 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028965 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028969 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028981 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028986 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028988 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028991 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028995 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.028997 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029000 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029003 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029005 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029012 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029015 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029019 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.034047 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029021 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.029024 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030264 2570 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030274 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030280 2570 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030284 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030289 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030292 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030296 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030300 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030303 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030306 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030310 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030313 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030316 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030319 2570 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030322 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030324 2570 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030327 2570 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030330 2570 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030335 2570 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030338 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030341 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030344 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:30:58.034485 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030348 2570 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030352 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030355 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030358 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030361 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030364 2570 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030367 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030372 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030375 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030378 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030382 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030385 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030388 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030391 2570 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030394 2570 flags.go:64] FLAG: --enable-server="true" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030397 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030401 2570 flags.go:64] FLAG: --event-burst="100" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030404 2570 flags.go:64] FLAG: --event-qps="50" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030407 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030410 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030412 2570 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030416 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030419 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030422 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030426 2570 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:30:58.035099 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030429 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030432 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030435 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030438 2570 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030441 2570 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030444 2570 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030447 2570 flags.go:64] FLAG: --feature-gates="" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030456 2570 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030459 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030462 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030465 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030468 2570 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030471 2570 flags.go:64] FLAG: --help="false" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030474 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030479 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030482 2570 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030485 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030488 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030491 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030494 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030497 2570 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030500 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030502 2570 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:30:58.035703 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030505 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030508 2570 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030511 2570 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030514 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030517 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030520 2570 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030523 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030525 2570 flags.go:64] FLAG: --lock-file="" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030528 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030531 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030534 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030540 2570 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030542 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030545 2570 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030548 2570 flags.go:64] FLAG: --logging-format="text" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030551 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030554 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030557 2570 flags.go:64] FLAG: --manifest-url="" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030560 2570 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030564 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030567 2570 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030571 2570 flags.go:64] FLAG: --max-pods="110" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030573 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030577 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030581 2570 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:30:58.036271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030584 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030587 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030590 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030592 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030599 2570 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030602 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030605 2570 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030608 2570 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030611 2570 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030613 2570 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030616 2570 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030619 2570 flags.go:64] FLAG: --port="10250" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030622 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030625 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09e70bc2ab84269f0" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030628 2570 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030631 2570 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030634 2570 flags.go:64] FLAG: --register-node="true" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030637 2570 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030640 2570 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030643 2570 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030646 2570 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030648 2570 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030652 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030655 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030658 2570 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030661 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:30:58.036838 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030663 2570 flags.go:64] FLAG: --runonce="false" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030666 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030670 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030673 2570 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030677 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030680 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030683 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030686 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030689 2570 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030692 2570 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030694 2570 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030697 2570 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030700 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030703 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030706 2570 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030709 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030720 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030724 2570 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030727 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030732 2570 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030734 2570 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030737 2570 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030740 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030743 2570 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030746 2570 flags.go:64] FLAG: --v="2" Oct 30 18:30:58.037447 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030750 2570 flags.go:64] FLAG: --version="false" Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030753 2570 flags.go:64] FLAG: --vmodule="" Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030757 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.030762 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030935 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030939 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030943 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030946 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030962 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030966 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030970 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030973 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030976 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030979 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030982 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030985 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030988 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030990 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030993 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030995 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.038067 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.030999 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031001 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031004 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031006 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031009 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031011 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031014 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031017 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031019 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031022 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031025 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031027 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031030 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031032 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031035 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031039 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031041 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031044 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031046 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.038519 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031049 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031051 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031054 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031056 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031059 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031061 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031064 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031067 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031070 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031072 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031075 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031077 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031080 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031082 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031085 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031089 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031093 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031096 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031098 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.039002 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031101 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031104 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031106 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031109 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031111 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031114 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031116 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031119 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031121 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031125 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031128 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031130 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031133 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031136 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031138 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031140 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031143 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031146 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031148 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031151 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.039454 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031153 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031156 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031160 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031162 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031165 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031167 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031170 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031172 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031175 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031177 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031180 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031183 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031185 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031187 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031190 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031192 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031195 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031197 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031200 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031202 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.039922 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031205 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031208 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031211 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031214 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031216 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031219 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031223 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031226 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031230 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031233 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031235 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031238 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.031265 2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031300 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031305 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.040498 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031309 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031312 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031315 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031318 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031320 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031323 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031326 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031328 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031331 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031333 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031336 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031338 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031341 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031343 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031346 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031348 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031351 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031353 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031357 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031359 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.040850 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031362 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031365 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031367 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031370 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031373 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031376 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031379 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031381 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031384 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031386 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031389 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031391 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031394 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031397 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031401 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031405 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031408 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031411 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031414 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.041313 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031416 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031419 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031422 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031424 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031427 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031429 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031432 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031435 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031437 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031440 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031442 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031446 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031449 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031451 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031454 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031457 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031459 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031463 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031465 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.041746 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031468 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031470 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031473 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031475 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031478 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031481 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031484 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031486 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031489 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031492 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031494 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031497 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031500 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031502 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031505 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031507 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031510 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031512 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031515 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.042177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031517 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031520 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031522 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031525 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031527 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031530 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031533 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031536 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031538 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031541 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031543 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031547 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031549 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031554 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031558 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031561 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031564 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031567 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031570 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.042600 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031574 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031577 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031580 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031583 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031585 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031588 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031590 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.031593 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.037739 2570 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.037752 2570 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.038284 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.038290 2570 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038345 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038351 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038354 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038357 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038360 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038363 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038365 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038368 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.043078 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038371 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038373 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038376 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038378 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038381 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038385 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038388 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038391 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038394 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038396 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038399 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038401 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038404 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038406 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038408 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038412 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038415 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038418 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038420 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.043574 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038422 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038425 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038428 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038431 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038433 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038436 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038438 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038441 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038445 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038449 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038452 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038455 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038457 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038460 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038463 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038466 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038468 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038470 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038473 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.044084 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038475 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038478 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038480 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038484 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038486 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038489 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038491 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038494 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038497 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038499 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038503 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038506 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038508 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038510 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038513 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038516 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038519 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038521 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038524 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038527 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.044521 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038529 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038532 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038534 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038537 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038540 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038542 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038544 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038547 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038550 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038552 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038555 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038557 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038560 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038562 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038565 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038568 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038570 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038573 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038575 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.044972 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038578 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038580 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038583 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038585 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038595 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038598 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038600 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038603 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038605 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038608 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038610 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038613 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038615 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038617 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038620 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038622 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038625 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038627 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038630 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038632 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.045419 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038635 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.038663 2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038818 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038824 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038826 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038829 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038832 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038834 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038837 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038840 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038842 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038845 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038847 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038850 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038852 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038855 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.045878 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038857 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038861 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038863 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038866 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038870 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038873 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038876 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038879 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038882 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038885 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038888 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038891 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038893 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038896 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038899 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038901 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038904 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038906 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038909 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.046262 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038911 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038914 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038916 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038919 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038922 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038925 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038927 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038930 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038932 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038935 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038937 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038940 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038942 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038945 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038948 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038965 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038970 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038972 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038975 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038978 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.046710 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038980 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038983 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038985 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038988 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038990 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038993 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.038997 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039001 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039003 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039007 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039010 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039013 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039016 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039018 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039021 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039023 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039026 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039028 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039031 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.047211 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039033 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039036 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039038 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039040 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039043 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039045 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039048 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039050 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039053 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039056 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039058 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039061 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039063 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039066 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039068 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039071 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039073 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039078 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039081 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.047652 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039084 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039086 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039089 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039091 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039094 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039096 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039099 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039102 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039104 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039107 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039110 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039112 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039115 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039117 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039120 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.039144 2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:58.048144 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039177 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039181 2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039184 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039187 2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039190 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039193 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039196 2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039199 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039201 2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039204 2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039206 2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039209 2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039212 2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039214 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039217 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039221 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039224 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039226 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039229 2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:58.048512 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039232 2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039234 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039237 2570 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039240 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039243 2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039246 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039248 2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039251 2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039254 2570 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039257 2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039259 2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039261 2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039264 2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039267 2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039269 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039273 2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039277 2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039280 2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039283 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:58.048933 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039285 2570 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039288 2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039291 2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039294 2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039296 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039298 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039301 2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039303 2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039306 2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039310 2570 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039313 2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039315 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039318 2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039320 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039323 2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039325 2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039328 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039330 2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039333 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:58.049409 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039336 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039338 2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039341 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039343 2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039346 2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039349 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039351 2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039354 2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039357 2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039359 2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039361 2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039364 2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039366 2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039369 2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039371 2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039374 2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039377 2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039380 2570 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039382 2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039385 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:58.049867 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039387 2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039390 2570 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039394 2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039396 2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039399 2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039402 2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039404 2570 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039407 2570 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039409 2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039412 2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039414 2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039417 2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039419 2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039422 2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039424 2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039427 2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039430 2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039432 2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039435 2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039437 2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:58.050317 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039440 2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039442 2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039447 2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039450 2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039453 2570 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039456 2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039461 2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039464 2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:58.039466 2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.039516 2570 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.042148 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.043069 2570 server.go:1014] "Starting client certificate rotation" Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.043166 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.043198 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:30:58.050840 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.045622 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:30:58.053651 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.053631 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:30:58.065035 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.065018 2570 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:30:58.068177 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.068156 2570 log.go:25] "Validated CRI v1 image API" Oct 30 18:30:58.069007 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.068983 2570 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:30:58.073921 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.073900 2570 fs.go:135] Filesystem UUIDs: map[2c5434fc-ef10-4e71-8d54-2813fd66abbd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b7137dc7-52cf-4708-a762-a51e82706c5c:/dev/nvme0n1p3] Oct 30 18:30:58.074009 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.073920 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 18:30:58.079074 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.078932 2570 manager.go:217] Machine: {Timestamp:2026-10-30 18:30:58.077520933 +0000 UTC m=+0.314589517 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3195090 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f41c4c54a2a7516e3d32610d7b3af SystemUUID:ec2f41c4-c54a-2a75-16e3-d32610d7b3af BootID:b6958c44-a391-4f68-a13b-c624e3d89f67 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c7:c7:90:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c7:c7:90:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:eb:c6:58:89:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:30:58.079074 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.079068 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:30:58.079236 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.079201 2570 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:30:58.082316 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.082289 2570 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:30:58.082437 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.082317 2570 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-103.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:30:58.082981 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.082971 2570 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:30:58.083017 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.082982 2570 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:30:58.083017 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.083013 2570 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:30:58.083066 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.083026 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:30:58.083861 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.083852 2570 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:30:58.088471 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.088458 2570 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:30:58.088581 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.088572 2570 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:30:58.089349 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.089340 2570 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:30:58.089381 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.089353 2570 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:30:58.089381 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.089363 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:30:58.089381 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.089371 2570 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:30:58.089381 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.089379 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:30:58.089715 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.089702 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:30:58.091919 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.091887 2570 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:30:58.096561 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.096549 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:30:58.097331 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097319 2570 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:30:58.097372 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097348 2570 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:30:58.097489 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097478 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:30:58.097520 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097495 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:30:58.097520 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097501 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:30:58.097520 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097507 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:30:58.097520 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097512 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:30:58.097520 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.097518 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:30:58.098097 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098088 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:30:58.098126 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098098 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:30:58.098126 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098105 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:30:58.098126 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098111 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:30:58.098126 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098119 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:30:58.098278 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098128 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:30:58.098668 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098660 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:30:58.098699 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.098672 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:30:58.102305 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.102290 2570 server.go:1263] "Started kubelet" Oct 30 18:30:58.102477 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.102418 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:30:58.102541 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.102384 2570 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:30:58.102592 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.102559 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:30:58.103134 ip-10-0-136-103 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:30:58.107211 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.106806 2570 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:30:58.108748 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.108730 2570 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:30:58.112457 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.112435 2570 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 18:30:58.113625 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.113604 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:30:58.114035 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.114019 2570 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:30:58.114184 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.114165 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.114648 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.114596 2570 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:30:58.114737 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.114652 2570 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:30:58.114737 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.114620 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:30:58.114831 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.114770 2570 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:30:58.114831 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.114779 2570 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:30:58.115038 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115026 2570 factory.go:153] Registering CRI-O factory Oct 30 18:30:58.115090 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115041 2570 factory.go:223] Registration of the crio container factory successfully Oct 30 18:30:58.115128 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115110 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:30:58.115128 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115117 2570 factory.go:55] Registering systemd factory Oct 30 18:30:58.115128 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115124 2570 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:30:58.115238 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115139 2570 factory.go:103] Registering Raw factory Oct 30 18:30:58.115238 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.115148 2570 manager.go:1199] Started watching for new ooms in manager Oct 30 18:30:58.116080 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.116064 2570 manager.go:319] Starting recovery of all containers Oct 30 18:30:58.128210 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.128191 2570 manager.go:324] Recovery completed Oct 30 18:30:58.134438 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.134420 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.137398 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.137359 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.137482 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.137421 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.137482 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.137439 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.138081 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.138066 2570 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:30:58.138081 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.138079 2570 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:30:58.138188 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.138100 2570 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:30:58.138188 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.138073 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.139128 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.139110 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.139200 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.139145 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.139200 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.139157 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.140443 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.140429 2570 policy_none.go:50] "Start" Oct 30 18:30:58.140443 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.140445 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:30:58.140563 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.140458 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:30:58.142548 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.142530 2570 policy_none.go:44] "Start" Oct 30 18:30:58.142643 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.142568 2570 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 18:30:58.142643 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.142582 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-103.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 18:30:58.143663 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.142660 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-103.ec2.internal.18e3623565adc412 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-103.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-103.ec2.internal,},FirstTimestamp:2026-10-30 18:30:58.10226485 +0000 UTC m=+0.339333434,LastTimestamp:2026-10-30 18:30:58.10226485 +0000 UTC m=+0.339333434,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-103.ec2.internal,}" Oct 30 18:30:58.144390 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.144360 2570 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:30:58.144943 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.144922 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7v5hr" Oct 30 18:30:58.157427 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.157405 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7v5hr" Oct 30 18:30:58.158552 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.158525 2570 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 18:30:58.158617 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.158525 2570 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 18:30:58.168004 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.167984 2570 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 18:30:58.168004 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.167919 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-103.ec2.internal.18e3623567c5e0f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-103.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-103.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-103.ec2.internal,},FirstTimestamp:2026-10-30 18:30:58.137399543 +0000 UTC m=+0.374468132,LastTimestamp:2026-10-30 18:30:58.137399543 +0000 UTC m=+0.374468132,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-103.ec2.internal,}" Oct 30 18:30:58.178614 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.178582 2570 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:30:58.181267 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181252 2570 manager.go:341] "Starting Device Plugin manager" Oct 30 18:30:58.181384 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.181288 2570 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:30:58.181384 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181296 2570 server.go:87] "Starting device plugin registration server" Oct 30 18:30:58.181559 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181546 2570 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:30:58.181602 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181562 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:30:58.181826 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181705 2570 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:30:58.181826 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181795 2570 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:30:58.181826 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.181802 2570 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:30:58.182314 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.182294 2570 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:30:58.182415 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.182336 2570 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.241928 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.241892 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:30:58.243057 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.243038 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:30:58.243119 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.243068 2570 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:30:58.243119 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.243094 2570 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:30:58.243218 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.243136 2570 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 18:30:58.259243 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.259224 2570 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:58.282693 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.282675 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.283663 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.283639 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.283767 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.283674 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.283767 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.283689 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.283767 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.283719 2570 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.304533 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.304507 2570 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.304636 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.304536 2570 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-103.ec2.internal\": node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.343442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.343410 2570 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal"] Oct 30 18:30:58.343567 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.343477 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.344386 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.344370 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.344436 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.344401 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.344436 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.344413 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.346545 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.346533 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.346709 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.346695 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.346753 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.346734 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.347385 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.347369 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.347474 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.347399 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.347474 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.347370 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.347474 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.347412 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.347474 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.347427 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.347474 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.347447 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.349456 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.349439 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.349539 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.349468 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:58.350379 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.350362 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:58.350442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.350395 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:58.350442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.350412 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:58.351630 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.351615 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.370150 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.370133 2570 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-103.ec2.internal\" not found" node="ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.372482 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.372464 2570 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-103.ec2.internal\" not found" node="ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.416237 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.416216 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e1514c45542b472144184b401445b5d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" (UID: \"1e1514c45542b472144184b401445b5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.416309 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.416243 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.416309 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.416259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.451854 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.451826 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.516795 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.516775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.516856 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.516805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.516856 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.516830 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e1514c45542b472144184b401445b5d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" (UID: \"1e1514c45542b472144184b401445b5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.516856 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.516847 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.516983 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.516868 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e1514c45542b472144184b401445b5d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" (UID: \"1e1514c45542b472144184b401445b5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.516983 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.516877 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.551946 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.551897 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.652336 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.652318 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.673197 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.673176 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.675331 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:58.675311 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:30:58.753216 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.753026 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.853501 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.853447 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:58.953969 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:58.953929 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.042444 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.042421 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 18:30:59.042839 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.042537 2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 18:30:59.054702 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:59.054683 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.155216 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:59.155194 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.162377 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.162348 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:25:58 +0000 UTC" deadline="2028-07-25 20:55:10.632605426 +0000 UTC" Oct 30 18:30:59.162473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.162385 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15218h24m11.470234012s" Oct 30 18:30:59.255725 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:59.255696 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.327658 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:59.327618 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1514c45542b472144184b401445b5d.slice/crio-926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024 WatchSource:0}: Error finding container 926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024: Status 404 returned error can't find the container with id 926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024 Oct 30 18:30:59.328160 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:30:59.328139 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4846e669c7fd95841df1580885fb26.slice/crio-424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112 WatchSource:0}: Error finding container 424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112: Status 404 returned error can't find the container with id 424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112 Oct 30 18:30:59.332511 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.332497 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:30:59.337918 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.337898 2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:59.338953 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.338937 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:59.339037 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.338966 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:59.339037 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.338992 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:59.356496 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:59.356476 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.457336 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:59.457258 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.529137 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.529112 2570 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:59.557656 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:30:59.557621 2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:30:59.613248 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.613221 2570 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:59.620783 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.620765 2570 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:59.715396 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.715318 2570 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:30:59.738146 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.738109 2570 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:30:59.739090 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.739064 2570 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:30:59.765226 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:30:59.765200 2570 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:31:00.090027 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.089982 2570 apiserver.go:52] "Watching apiserver" Oct 30 18:31:00.115362 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.114734 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 18:31:00.124047 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.124022 2570 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:31:00.124407 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.124373 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-v2hkd","openshift-dns/node-resolver-kbnmf","openshift-multus/multus-j4m82","kube-system/global-pull-secret-syncer-9vhcp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh","openshift-network-operator/iptables-alerter-b9875","openshift-ovn-kubernetes/ovnkube-node-llt5l","kube-system/konnectivity-agent-t9b8c","openshift-image-registry/node-ca-z8j6q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal","openshift-multus/multus-additional-cni-plugins-jxr57","openshift-multus/network-metrics-daemon-29d22","openshift-network-diagnostics/network-check-target-kvgwg","kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal"] Oct 30 18:31:00.127197 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.127169 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.133491 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.133235 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:00.133491 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.133308 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:00.136291 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.136272 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.138623 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.138606 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.138718 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.138658 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:00.141823 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.141803 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.144252 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.144232 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.146342 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.145842 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:31:00.146487 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.146466 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8srjr\"" Oct 30 18:31:00.146551 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.146508 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:31:00.146666 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.146646 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:31:00.146935 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.146918 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.147185 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.146971 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.147185 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.147049 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:31:00.147185 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.147135 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.149304 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.149274 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.152598 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.152580 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.155630 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.155613 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.158519 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.158500 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.161193 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.161125 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.161275 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.161192 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:00.183209 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.183182 2570 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:31:00.183337 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.183268 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ntwsw" Oct 30 18:31:00.194925 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.194904 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ntwsw" Oct 30 18:31:00.216803 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.216784 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:31:00.226652 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226627 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-netns\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.226759 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226667 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.226759 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226692 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-modprobe-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.226759 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-conf\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.226759 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226737 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xhxm\" (UniqueName: \"kubernetes.io/projected/514fb142-256f-466f-aefb-dc0fbc1ce334-kube-api-access-5xhxm\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226764 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-systemd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226779 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-script-lib\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226810 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226830 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/219a3597-95e6-4836-9f38-e8091e0f0b35-host-slash\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226852 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-systemd\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226872 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-sys-fs\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226893 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvx8s\" (UniqueName: \"kubernetes.io/projected/7eb17ce4-c80f-476b-8682-c1a329bd82fb-kube-api-access-bvx8s\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.226924 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226916 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-log-socket\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-kubelet-config\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226973 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/219a3597-95e6-4836-9f38-e8091e0f0b35-iptables-alerter-script\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.226998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsfss\" (UniqueName: \"kubernetes.io/projected/219a3597-95e6-4836-9f38-e8091e0f0b35-kube-api-access-fsfss\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227016 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227030 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysconfig\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227046 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-cnibin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227060 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-multus\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227090 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d54g6\" (UniqueName: \"kubernetes.io/projected/71968925-61e2-439b-8d7f-7d9184c7032b-kube-api-access-d54g6\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227103 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-system-cni-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227117 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/422a1ecf-21d1-433f-8ede-d9de5a5940b7-serviceca\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227129 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-etc-kubernetes\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227149 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-systemd-units\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227177 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-config\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227207 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-dbus\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-var-lib-kubelet\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227258 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227282 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-socket-dir-parent\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb17ce4-c80f-476b-8682-c1a329bd82fb-tmp-dir\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227328 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-run\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227379 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-slash\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227404 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227425 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-tmp\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227445 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227473 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-daemon-config\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227489 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6364988c-a134-41de-9d5f-38728dd0808c-agent-certs\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227504 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-cnibin\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227521 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-etc-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227544 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-ovn\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.227705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227572 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5h4s\" (UniqueName: \"kubernetes.io/projected/efb3d9bb-411e-4587-b807-3054f555450a-kube-api-access-t5h4s\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227607 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-host\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227621 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvmln\" (UniqueName: \"kubernetes.io/projected/8b5c8f73-4e7d-4e68-939f-288190998d5a-kube-api-access-bvmln\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227641 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqg5r\" (UniqueName: \"kubernetes.io/projected/a3635fec-23c3-49ab-8606-9803e62d959c-kube-api-access-zqg5r\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227660 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-env-overrides\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efb3d9bb-411e-4587-b807-3054f555450a-ovn-node-metrics-cert\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227713 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-tuned\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227737 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-hostroot\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-conf-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-socket-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-registration-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227825 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-var-lib-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227843 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422a1ecf-21d1-433f-8ede-d9de5a5940b7-host\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227871 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-kubernetes\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.228289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227896 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-system-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-bin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-sys\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.227986 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-lib-modules\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-cni-binary-copy\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-netns\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-multus-certs\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228283 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-device-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228315 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7eb17ce4-c80f-476b-8682-c1a329bd82fb-hosts-file\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228358 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6364988c-a134-41de-9d5f-38728dd0808c-konnectivity-ca\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228393 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-kubelet\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228445 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-kubelet\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228490 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228526 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcrsm\" (UniqueName: \"kubernetes.io/projected/7d565d10-f9d0-4450-a666-2c0805d53629-kube-api-access-xcrsm\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228596 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-node-log\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-bin\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.229019 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228665 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-netd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.229733 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjktc\" (UniqueName: \"kubernetes.io/projected/422a1ecf-21d1-433f-8ede-d9de5a5940b7-kube-api-access-gjktc\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.229733 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-k8s-cni-cncf-io\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.229733 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.228797 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.247483 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247444 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:31:00.247614 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247444 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:31:00.247733 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247462 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.247733 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247482 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:31:00.247733 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247497 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:31:00.247887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247499 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:31:00.247887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247737 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.247887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247766 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.247887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247861 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247909 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247942 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247992 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kkmm\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.247999 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248015 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsmgf\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248028 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248057 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248064 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:00.248076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248002 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248126 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248171 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xfxj4\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248206 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248303 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ghgqv\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248352 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248395 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:31:00.248473 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248465 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ft7c\"" Oct 30 18:31:00.248781 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248566 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtzwt\"" Oct 30 18:31:00.248781 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248655 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k7jl9\"" Oct 30 18:31:00.248781 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248712 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:31:00.248781 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248769 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-56ppb\"" Oct 30 18:31:00.249039 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.248979 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" event={"ID":"2d4846e669c7fd95841df1580885fb26","Type":"ContainerStarted","Data":"424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112"} Oct 30 18:31:00.250284 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.250260 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" event={"ID":"1e1514c45542b472144184b401445b5d","Type":"ContainerStarted","Data":"926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024"} Oct 30 18:31:00.330041 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/422a1ecf-21d1-433f-8ede-d9de5a5940b7-serviceca\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.330228 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330049 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-etc-kubernetes\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.330228 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-systemd-units\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.330228 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330106 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-config\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.330228 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330200 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-dbus\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.330428 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330215 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-etc-kubernetes\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.330428 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330296 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-systemd-units\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.330428 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330308 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-var-lib-kubelet\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.330563 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330226 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-var-lib-kubelet\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.330563 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330480 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-dbus\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.330563 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.330563 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330530 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-socket-dir-parent\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.330563 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb17ce4-c80f-476b-8682-c1a329bd82fb-tmp-dir\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330580 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330606 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330615 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-run\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330629 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-socket-dir-parent\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330649 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-slash\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330662 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-run\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330694 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-slash\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330724 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.330756 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-tmp\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330763 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330782 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-daemon-config\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6364988c-a134-41de-9d5f-38728dd0808c-agent-certs\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-cnibin\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330844 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-cnibin\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330879 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.330885 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331058 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/422a1ecf-21d1-433f-8ede-d9de5a5940b7-serviceca\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.331379 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331303 2570 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:31:00.331460 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-etc-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331495 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-ovn\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331527 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331508 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5h4s\" (UniqueName: \"kubernetes.io/projected/efb3d9bb-411e-4587-b807-3054f555450a-kube-api-access-t5h4s\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331574 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.331574 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331564 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-host\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.331659 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331566 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-etc-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331659 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331597 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-ovn\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331659 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331612 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-config\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.331791 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331671 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-host\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.331791 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvmln\" (UniqueName: \"kubernetes.io/projected/8b5c8f73-4e7d-4e68-939f-288190998d5a-kube-api-access-bvmln\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.331791 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331755 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqg5r\" (UniqueName: \"kubernetes.io/projected/a3635fec-23c3-49ab-8606-9803e62d959c-kube-api-access-zqg5r\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.331791 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331713 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.331791 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331785 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-env-overrides\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efb3d9bb-411e-4587-b807-3054f555450a-ovn-node-metrics-cert\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331873 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331896 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-tuned\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-hostroot\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331966 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-conf-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.331990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-socket-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-registration-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.332046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332039 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-var-lib-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332063 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422a1ecf-21d1-433f-8ede-d9de5a5940b7-host\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-kubernetes\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332143 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-system-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-bin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-sys\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332204 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-registration-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332188 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb17ce4-c80f-476b-8682-c1a329bd82fb-tmp-dir\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332267 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-hostroot\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332300 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332307 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-var-lib-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332314 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-conf-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332329 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-env-overrides\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.332366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332356 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422a1ecf-21d1-433f-8ede-d9de5a5940b7-host\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.332393 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332418 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-socket-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332410 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-sys\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332477 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-system-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332513 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-lib-modules\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332552 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-bin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332482 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-kubernetes\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.332561 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:00.832532936 +0000 UTC m=+3.069601512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-cni-binary-copy\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332623 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-netns\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332674 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-netns\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332685 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-multus-certs\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-lib-modules\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-device-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332726 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-multus-certs\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.332973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332769 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-device-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332787 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7eb17ce4-c80f-476b-8682-c1a329bd82fb-hosts-file\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332814 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6364988c-a134-41de-9d5f-38728dd0808c-konnectivity-ca\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-kubelet\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332869 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7eb17ce4-c80f-476b-8682-c1a329bd82fb-hosts-file\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332871 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-kubelet\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332903 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-kubelet\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332913 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332944 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcrsm\" (UniqueName: \"kubernetes.io/projected/7d565d10-f9d0-4450-a666-2c0805d53629-kube-api-access-xcrsm\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.332987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-node-log\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.333015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-bin\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.333038 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-netd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.333065 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjktc\" (UniqueName: \"kubernetes.io/projected/422a1ecf-21d1-433f-8ede-d9de5a5940b7-kube-api-access-gjktc\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.333091 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-k8s-cni-cncf-io\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.333799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.333257 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-kubelet\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.336525 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.333111 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.336614 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-netns\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.336660 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336602 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.336660 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-modprobe-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.336738 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-conf\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.336738 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336697 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhxm\" (UniqueName: \"kubernetes.io/projected/514fb142-256f-466f-aefb-dc0fbc1ce334-kube-api-access-5xhxm\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.336738 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336726 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-systemd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.336836 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336767 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.336836 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336804 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-script-lib\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.336896 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:00.336896 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/219a3597-95e6-4836-9f38-e8091e0f0b35-host-slash\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.336978 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336899 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-systemd\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.336978 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336923 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-sys-fs\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.337087 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.336977 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvx8s\" (UniqueName: \"kubernetes.io/projected/7eb17ce4-c80f-476b-8682-c1a329bd82fb-kube-api-access-bvx8s\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.337087 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337027 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-log-socket\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.337179 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337083 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-kubelet-config\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.337179 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/219a3597-95e6-4836-9f38-e8091e0f0b35-iptables-alerter-script\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.337179 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337146 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsfss\" (UniqueName: \"kubernetes.io/projected/219a3597-95e6-4836-9f38-e8091e0f0b35-kube-api-access-fsfss\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.337179 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337173 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.337312 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337217 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysconfig\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.337312 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337232 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6364988c-a134-41de-9d5f-38728dd0808c-konnectivity-ca\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.337312 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-cnibin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.337392 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337318 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-multus\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.337392 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337325 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-cnibin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.337392 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337355 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.337392 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337377 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d54g6\" (UniqueName: \"kubernetes.io/projected/71968925-61e2-439b-8d7f-7d9184c7032b-kube-api-access-d54g6\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.337498 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337390 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-netns\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.337498 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337396 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-system-cni-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.337498 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337440 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.337580 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337499 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-systemd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.337580 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337567 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.337988 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.337947 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efb3d9bb-411e-4587-b807-3054f555450a-ovn-node-metrics-cert\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.338166 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.338150 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-script-lib\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.338253 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-kubelet-config\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.338746 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/219a3597-95e6-4836-9f38-e8091e0f0b35-iptables-alerter-script\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339109 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-daemon-config\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.339147 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339156 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-tmp\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339222 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-bin\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.339242 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:00.839196719 +0000 UTC m=+3.076265294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339295 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-node-log\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339338 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysconfig\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339386 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-log-socket\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339424 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-conf\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339465 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-cni-binary-copy\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339649 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-multus\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339666 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/219a3597-95e6-4836-9f38-e8091e0f0b35-host-slash\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339718 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-system-cni-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339725 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-systemd\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.340233 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339727 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.341014 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339812 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-sys-fs\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.341014 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.339930 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6364988c-a134-41de-9d5f-38728dd0808c-agent-certs\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.341014 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.340208 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-tuned\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.341014 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.340340 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-netd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.341014 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.340500 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-k8s-cni-cncf-io\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.341014 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.340642 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-modprobe-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.341601 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.341559 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.341697 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.341617 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.360874 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.360849 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5h4s\" (UniqueName: \"kubernetes.io/projected/efb3d9bb-411e-4587-b807-3054f555450a-kube-api-access-t5h4s\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.361000 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.360847 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvmln\" (UniqueName: \"kubernetes.io/projected/8b5c8f73-4e7d-4e68-939f-288190998d5a-kube-api-access-bvmln\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.365866 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.365824 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsfss\" (UniqueName: \"kubernetes.io/projected/219a3597-95e6-4836-9f38-e8091e0f0b35-kube-api-access-fsfss\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.366278 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.366257 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhxm\" (UniqueName: \"kubernetes.io/projected/514fb142-256f-466f-aefb-dc0fbc1ce334-kube-api-access-5xhxm\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.368308 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.368256 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcrsm\" (UniqueName: \"kubernetes.io/projected/7d565d10-f9d0-4450-a666-2c0805d53629-kube-api-access-xcrsm\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.369406 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.369380 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:00.369406 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.369409 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:00.369537 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.369423 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:00.369537 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.369493 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:31:00.86947363 +0000 UTC m=+3.106542220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:00.372396 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.372371 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d54g6\" (UniqueName: \"kubernetes.io/projected/71968925-61e2-439b-8d7f-7d9184c7032b-kube-api-access-d54g6\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.372652 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.372633 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvx8s\" (UniqueName: \"kubernetes.io/projected/7eb17ce4-c80f-476b-8682-c1a329bd82fb-kube-api-access-bvx8s\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.374813 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.374793 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqg5r\" (UniqueName: \"kubernetes.io/projected/a3635fec-23c3-49ab-8606-9803e62d959c-kube-api-access-zqg5r\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.377095 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.377075 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjktc\" (UniqueName: \"kubernetes.io/projected/422a1ecf-21d1-433f-8ede-d9de5a5940b7-kube-api-access-gjktc\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.435683 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.435654 2570 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:31:00.455476 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.455450 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j4m82" Oct 30 18:31:00.461079 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.461054 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:00.468177 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.468147 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514fb142_256f_466f_aefb_dc0fbc1ce334.slice/crio-ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4 WatchSource:0}: Error finding container ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4: Status 404 returned error can't find the container with id ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4 Oct 30 18:31:00.470973 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.470930 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:31:00.471291 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.471268 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb3d9bb_411e_4587_b807_3054f555450a.slice/crio-93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609 WatchSource:0}: Error finding container 93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609: Status 404 returned error can't find the container with id 93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609 Oct 30 18:31:00.472418 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.472322 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:31:00.477853 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.477836 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:31:00.483871 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.483855 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:31:00.483946 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.483922 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71968925_61e2_439b_8d7f_7d9184c7032b.slice/crio-c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a WatchSource:0}: Error finding container c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a: Status 404 returned error can't find the container with id c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a Oct 30 18:31:00.485857 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.485837 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219a3597_95e6_4836_9f38_e8091e0f0b35.slice/crio-aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16 WatchSource:0}: Error finding container aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16: Status 404 returned error can't find the container with id aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16 Oct 30 18:31:00.491076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.491052 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:31:00.491360 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.491333 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422a1ecf_21d1_433f_8ede_d9de5a5940b7.slice/crio-86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f WatchSource:0}: Error finding container 86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f: Status 404 returned error can't find the container with id 86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f Oct 30 18:31:00.497287 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.497262 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5c8f73_4e7d_4e68_939f_288190998d5a.slice/crio-e4ebc19390ba54daa88ffbe0cc55ace12e530bd7523b8206808c6e6b39ef490e WatchSource:0}: Error finding container e4ebc19390ba54daa88ffbe0cc55ace12e530bd7523b8206808c6e6b39ef490e: Status 404 returned error can't find the container with id e4ebc19390ba54daa88ffbe0cc55ace12e530bd7523b8206808c6e6b39ef490e Oct 30 18:31:00.498463 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.498442 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:00.502495 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.502475 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb17ce4_c80f_476b_8682_c1a329bd82fb.slice/crio-a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d WatchSource:0}: Error finding container a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d: Status 404 returned error can't find the container with id a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d Oct 30 18:31:00.508903 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.508883 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6364988c_a134_41de_9d5f_38728dd0808c.slice/crio-990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be WatchSource:0}: Error finding container 990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be: Status 404 returned error can't find the container with id 990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be Oct 30 18:31:00.519276 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.519246 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:31:00.529290 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:00.529263 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3635fec_23c3_49ab_8606_9803e62d959c.slice/crio-7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb WatchSource:0}: Error finding container 7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb: Status 404 returned error can't find the container with id 7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb Oct 30 18:31:00.840914 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.840891 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:00.841043 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.840967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:00.841043 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.841040 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:00.841127 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.841054 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:00.841127 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.841096 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:01.841080529 +0000 UTC m=+4.078149116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:00.841127 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.841109 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:01.841103084 +0000 UTC m=+4.078171656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:00.941749 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:00.941713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:00.942103 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.941939 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:00.942103 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.941982 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:00.942103 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.941996 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:00.942103 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:00.942067 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:31:01.942034283 +0000 UTC m=+4.179102861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:01.198044 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.197791 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:26:00 +0000 UTC" deadline="2028-04-27 13:53:38.359573653 +0000 UTC" Oct 30 18:31:01.198044 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.197849 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13075h22m37.161729145s" Oct 30 18:31:01.265904 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.265864 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" event={"ID":"71968925-61e2-439b-8d7f-7d9184c7032b","Type":"ContainerStarted","Data":"c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a"} Oct 30 18:31:01.276865 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.276833 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609"} Oct 30 18:31:01.284521 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.284429 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j4m82" event={"ID":"514fb142-256f-466f-aefb-dc0fbc1ce334","Type":"ContainerStarted","Data":"ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4"} Oct 30 18:31:01.301191 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.301162 2570 generic.go:356] "Generic (PLEG): container finished" podID="2d4846e669c7fd95841df1580885fb26" containerID="6f9cad31820d824adf2709d2029330d902a5088e279780787e3572a8a4c9ac6e" exitCode=0 Oct 30 18:31:01.301615 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.301430 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" event={"ID":"2d4846e669c7fd95841df1580885fb26","Type":"ContainerDied","Data":"6f9cad31820d824adf2709d2029330d902a5088e279780787e3572a8a4c9ac6e"} Oct 30 18:31:01.304807 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.304781 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t9b8c" event={"ID":"6364988c-a134-41de-9d5f-38728dd0808c","Type":"ContainerStarted","Data":"990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be"} Oct 30 18:31:01.310524 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.310500 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" event={"ID":"8b5c8f73-4e7d-4e68-939f-288190998d5a","Type":"ContainerStarted","Data":"e4ebc19390ba54daa88ffbe0cc55ace12e530bd7523b8206808c6e6b39ef490e"} Oct 30 18:31:01.326919 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.326899 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z8j6q" event={"ID":"422a1ecf-21d1-433f-8ede-d9de5a5940b7","Type":"ContainerStarted","Data":"86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f"} Oct 30 18:31:01.339940 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.339657 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b9875" event={"ID":"219a3597-95e6-4836-9f38-e8091e0f0b35","Type":"ContainerStarted","Data":"aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16"} Oct 30 18:31:01.344705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.342700 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" event={"ID":"1e1514c45542b472144184b401445b5d","Type":"ContainerStarted","Data":"e1965571488baf76fc984ff2f125c81e8747cda4f655b895757e8f0c6a984e4d"} Oct 30 18:31:01.351392 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.351329 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerStarted","Data":"7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb"} Oct 30 18:31:01.353793 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.353737 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kbnmf" event={"ID":"7eb17ce4-c80f-476b-8682-c1a329bd82fb","Type":"ContainerStarted","Data":"a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d"} Oct 30 18:31:01.853420 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.853383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:01.853684 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.853469 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:01.853684 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.853604 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:01.853684 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.853667 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:03.853650957 +0000 UTC m=+6.090719536 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:01.854060 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.854041 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:01.854141 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.854093 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:03.854081786 +0000 UTC m=+6.091150358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:01.955055 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:01.954476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:01.955055 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.954640 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:01.955055 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.954658 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:01.955055 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.954671 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:01.955055 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:01.954723 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:31:03.954705989 +0000 UTC m=+6.191774564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:02.198686 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.198587 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:26:00 +0000 UTC" deadline="2028-06-28 21:50:30.609230028 +0000 UTC" Oct 30 18:31:02.198686 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.198627 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14571h19m28.410606448s" Oct 30 18:31:02.246565 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.246535 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:02.246725 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:02.246666 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:02.247504 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.247038 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:02.247504 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:02.247147 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:02.247504 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.247199 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:02.247504 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:02.247274 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:02.363340 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.363233 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" event={"ID":"2d4846e669c7fd95841df1580885fb26","Type":"ContainerStarted","Data":"e7735e8310a5348ccccf627708bc7d1876f024309ebd1b1463f776509ae2a953"} Oct 30 18:31:02.409925 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:02.409190 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" podStartSLOduration=3.40916687 podStartE2EDuration="3.40916687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:01.389585296 +0000 UTC m=+3.626653887" watchObservedRunningTime="2026-10-30 18:31:02.40916687 +0000 UTC m=+4.646235465" Oct 30 18:31:03.873137 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:03.872348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:03.873137 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:03.872433 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:03.873137 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.872570 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:03.873137 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.872630 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:07.872614154 +0000 UTC m=+10.109682730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:03.873137 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.872948 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:03.873137 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.873015 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:07.872998912 +0000 UTC m=+10.110067497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:03.974863 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:03.974292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:03.974863 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.974452 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:03.974863 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.974473 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:03.974863 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.974484 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:03.974863 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:03.974535 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:31:07.974518099 +0000 UTC m=+10.211586688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:04.246075 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:04.245059 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:04.246075 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:04.245222 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:04.246075 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:04.245719 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:04.246075 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:04.245809 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:04.246075 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:04.245884 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:04.246075 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:04.245974 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:04.377639 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:04.376829 2570 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="04c8adbc9ea1c42554e8be34436367ed412f0cfe22faf37da556ddf4c91de3c7" exitCode=0 Oct 30 18:31:04.377639 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:04.376884 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerDied","Data":"04c8adbc9ea1c42554e8be34436367ed412f0cfe22faf37da556ddf4c91de3c7"} Oct 30 18:31:04.458214 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:04.458159 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" podStartSLOduration=5.458139764 podStartE2EDuration="5.458139764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:02.409877352 +0000 UTC m=+4.646945946" watchObservedRunningTime="2026-10-30 18:31:04.458139764 +0000 UTC m=+6.695208360" Oct 30 18:31:05.385939 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:05.385902 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t9b8c" event={"ID":"6364988c-a134-41de-9d5f-38728dd0808c","Type":"ContainerStarted","Data":"071d01f51f42c0e5a08510a5fe1313f3aee5593d8f636e7936392825c748d08d"} Oct 30 18:31:05.431030 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:05.430964 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t9b8c" podStartSLOduration=3.5138156929999997 podStartE2EDuration="7.430931441s" totalImagesPullingTime="3.917115748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:05.428793468 +0000 UTC m=+7.665862063" watchObservedRunningTime="2026-10-30 18:31:05.430931441 +0000 UTC m=+7.668000036" Oct 30 18:31:06.244369 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.244339 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:06.244369 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.244354 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:06.244598 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.244354 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:06.244598 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:06.244448 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:06.244598 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:06.244554 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:06.244795 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:06.244644 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:06.390799 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.390761 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" event={"ID":"71968925-61e2-439b-8d7f-7d9184c7032b","Type":"ContainerStarted","Data":"f812bb52f0506df7e06e3d04a21068dc594b27121133dd1fce1adb657281ea40"} Oct 30 18:31:06.395290 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.395233 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z8j6q" event={"ID":"422a1ecf-21d1-433f-8ede-d9de5a5940b7","Type":"ContainerStarted","Data":"e4208cba0fe02d7d9e6a099f26c45f9d31d3a48458010d6f43f07bc5142a91b9"} Oct 30 18:31:06.437484 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.437322 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z8j6q" podStartSLOduration=3.729575721 podStartE2EDuration="8.437301611s" totalImagesPullingTime="4.70772589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:06.436335263 +0000 UTC m=+8.673403859" watchObservedRunningTime="2026-10-30 18:31:06.437301611 +0000 UTC m=+8.674370205" Oct 30 18:31:06.995259 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:06.995221 2570 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:31:07.186660 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:07.186492 2570 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:31:06.995253204Z","UUID":"fe3101ef-ad36-434a-b7b7-8012b37428a8","Handler":null,"Name":"","Endpoint":""} Oct 30 18:31:07.188683 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:07.188454 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:31:07.188683 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:07.188479 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:31:07.401224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:07.401187 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" event={"ID":"71968925-61e2-439b-8d7f-7d9184c7032b","Type":"ContainerStarted","Data":"614476ca636e11fd8b9071ec1d4bea1450c5386503c097930b618bef4c0e8948"} Oct 30 18:31:07.904309 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:07.904269 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:07.904502 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:07.904365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:07.904566 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:07.904508 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:07.904606 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:07.904577 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.904558268 +0000 UTC m=+18.141626857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:07.905129 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:07.905068 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:07.905129 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:07.905139 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.905121717 +0000 UTC m=+18.142190302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:08.005249 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:08.005216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:08.005397 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.005383 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:08.005460 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.005406 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:08.005460 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.005426 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:08.005556 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.005479 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:31:16.005462918 +0000 UTC m=+18.242531509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:08.245097 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:08.244277 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:08.245097 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.244392 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:08.245097 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:08.244808 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:08.245097 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.244902 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:08.245423 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:08.245400 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:08.245546 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:08.245506 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:08.414221 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:08.411762 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j4m82" event={"ID":"514fb142-256f-466f-aefb-dc0fbc1ce334","Type":"ContainerStarted","Data":"3fccad25ec4cf3c431f318785d2cd37a7c70916c80f56b56bfa503f38ff27dff"} Oct 30 18:31:09.416559 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.416317 2570 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="1d17d86eb627d8cf937b93636214de85a0ac01fb909e0567f87ba0ee3e06d432" exitCode=0 Oct 30 18:31:09.417012 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.416552 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerDied","Data":"1d17d86eb627d8cf937b93636214de85a0ac01fb909e0567f87ba0ee3e06d432"} Oct 30 18:31:09.418788 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.418759 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kbnmf" event={"ID":"7eb17ce4-c80f-476b-8682-c1a329bd82fb","Type":"ContainerStarted","Data":"77ad341d1d2fa5fc54dcd26220b445b477af7380cb79100f57492153508a8359"} Oct 30 18:31:09.422109 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.422080 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" event={"ID":"71968925-61e2-439b-8d7f-7d9184c7032b","Type":"ContainerStarted","Data":"fc6b5f284d3da60a02557e00b21c6dde2c45ef74e5bc581db19132138c7541ef"} Oct 30 18:31:09.499053 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.498990 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-j4m82" podStartSLOduration=3.998820197 podStartE2EDuration="11.498973849s" totalImagesPullingTime="7.500153652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:08.46269889 +0000 UTC m=+10.699767487" watchObservedRunningTime="2026-10-30 18:31:09.498973849 +0000 UTC m=+11.736042440" Oct 30 18:31:09.550547 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.550481 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" podStartSLOduration=4.098309864 podStartE2EDuration="11.550461958s" totalImagesPullingTime="7.452152094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:09.550212978 +0000 UTC m=+11.787281572" watchObservedRunningTime="2026-10-30 18:31:09.550461958 +0000 UTC m=+11.787530552" Oct 30 18:31:09.960646 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.960613 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:09.961545 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:09.961521 2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:10.006792 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.006725 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-kbnmf" podStartSLOduration=4.188104902 podStartE2EDuration="12.006702285s" totalImagesPullingTime="7.818597383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:09.599483279 +0000 UTC m=+11.836551877" watchObservedRunningTime="2026-10-30 18:31:10.006702285 +0000 UTC m=+12.243770882" Oct 30 18:31:10.244132 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.244103 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:10.244225 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.244133 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:10.244264 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:10.244241 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:10.244314 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.244300 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:10.244439 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:10.244418 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:10.244563 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:10.244539 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:10.424889 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.424852 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" event={"ID":"8b5c8f73-4e7d-4e68-939f-288190998d5a","Type":"ContainerStarted","Data":"f64f872e0d2170847ac4b27714d939980eac5f458ca246a0e8d4f5e3330b4485"} Oct 30 18:31:10.426730 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.426699 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b9875" event={"ID":"219a3597-95e6-4836-9f38-e8091e0f0b35","Type":"ContainerStarted","Data":"1b1afde69d1c2b7fa586b55bb328345714b50f9b4eb6745067158cb8d1f59015"} Oct 30 18:31:10.429148 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.429108 2570 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="82f7b9e0735388290494b19c5f3c6a5f1409911322d5ef66cc4343c8ec565f20" exitCode=0 Oct 30 18:31:10.429285 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.429212 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerDied","Data":"82f7b9e0735388290494b19c5f3c6a5f1409911322d5ef66cc4343c8ec565f20"} Oct 30 18:31:10.472716 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.472636 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" podStartSLOduration=2.823735443 podStartE2EDuration="12.472623382s" totalImagesPullingTime="9.648887939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:10.472111099 +0000 UTC m=+12.709179695" watchObservedRunningTime="2026-10-30 18:31:10.472623382 +0000 UTC m=+12.709691973" Oct 30 18:31:10.515568 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:10.515512 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b9875" podStartSLOduration=4.7587633700000005 podStartE2EDuration="12.51549361s" totalImagesPullingTime="7.75673024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:10.514808864 +0000 UTC m=+12.751877457" watchObservedRunningTime="2026-10-30 18:31:10.51549361 +0000 UTC m=+12.752562205" Oct 30 18:31:11.433734 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:11.433686 2570 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="fd8a4741ef10e577a109c691d4a1bcf2f6156d4c0e84ecadefbe8cbbf67886ea" exitCode=0 Oct 30 18:31:11.434289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:11.433757 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerDied","Data":"fd8a4741ef10e577a109c691d4a1bcf2f6156d4c0e84ecadefbe8cbbf67886ea"} Oct 30 18:31:12.244311 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:12.244272 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:12.244311 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:12.244293 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:12.244524 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:12.244410 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:12.244524 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:12.244485 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:12.244600 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:12.244586 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:12.244670 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:12.244650 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:13.441870 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:13.441572 2570 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="1fe3a44fa44fa851895e96d9f2f2d3b1282096d6e466b04ad8ee996593b36913" exitCode=0 Oct 30 18:31:13.441870 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:13.441660 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerDied","Data":"1fe3a44fa44fa851895e96d9f2f2d3b1282096d6e466b04ad8ee996593b36913"} Oct 30 18:31:14.245816 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:14.244247 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:14.245816 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:14.244276 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:14.245816 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:14.244247 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:14.245816 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:14.244373 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:14.245816 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:14.244444 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:14.245816 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:14.244506 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:14.446903 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:14.446868 2570 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="a191e11436761d5bf68838165a6adf3e6c231ae2f02faa746177bbd648051f07" exitCode=0 Oct 30 18:31:14.447275 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:14.446926 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerDied","Data":"a191e11436761d5bf68838165a6adf3e6c231ae2f02faa746177bbd648051f07"} Oct 30 18:31:15.453071 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.452738 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jxr57" event={"ID":"a3635fec-23c3-49ab-8606-9803e62d959c","Type":"ContainerStarted","Data":"bf830d58dfc0d910b6360b6ef9ca39c98332257c3ce1d0e636f75fd79b1a8545"} Oct 30 18:31:15.456212 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.456183 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"3286615dedade2c32bb0d8397c755687f9d5ba86123ab512ef7e55e7d494800a"} Oct 30 18:31:15.456327 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.456247 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"54c9b9123c80d1cff9722b27fe6a669f69249f636995448db99b10f6ee04ef33"} Oct 30 18:31:15.456327 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.456276 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"c9f6829b7c3c120d6a25d5df63d5e41294f7adaff229fe13111b1600316f7344"} Oct 30 18:31:15.456327 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.456305 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"f15abad2f353577ff72526a8b0b889650f494ea345d7e07b120828f4ee34b59a"} Oct 30 18:31:15.456423 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.456333 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"161a8ba538e80b73412ce7fd897ea21aecfa482c1dfff8523ba83e7a687dc7b0"} Oct 30 18:31:15.456423 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.456359 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"fa098a266fa6fd1125df93ff2128051e734d4c11d6a876b4af926c506b3723a0"} Oct 30 18:31:15.508022 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.507978 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jxr57" podStartSLOduration=7.706112583 podStartE2EDuration="17.507947455s" totalImagesPullingTime="9.801834872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:15.507935558 +0000 UTC m=+17.745004151" watchObservedRunningTime="2026-10-30 18:31:15.507947455 +0000 UTC m=+17.745016048" Oct 30 18:31:15.963385 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.963349 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:15.963564 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:15.963395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:15.963564 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:15.963492 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:15.963564 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:15.963493 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:15.963564 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:15.963540 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:31:31.963528625 +0000 UTC m=+34.200597197 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:15.963564 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:15.963561 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:31.963547798 +0000 UTC m=+34.200616370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:16.064143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:16.064111 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:16.064301 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.064284 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:16.064356 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.064305 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:16.064356 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.064315 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:16.064416 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.064359 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:31:32.064347294 +0000 UTC m=+34.301415867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:16.243808 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:16.243730 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:16.243808 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:16.243777 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:16.244030 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.243890 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:16.244030 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.243923 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:16.244030 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:16.243994 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:16.244170 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:16.244091 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:17.463582 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:17.463500 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"675a22a91a2da32b7b4ab018aa21dddfdfbcdf5ea0017e3848b54bae384497fa"} Oct 30 18:31:18.245415 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:18.245384 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:18.245573 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:18.245482 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:18.245573 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:18.245524 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:18.245573 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:18.245515 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:18.245665 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:18.245614 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:18.245709 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:18.245691 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:19.471757 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:19.471722 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" event={"ID":"efb3d9bb-411e-4587-b807-3054f555450a","Type":"ContainerStarted","Data":"5bfacb8d68ff03a56ea01dff2ace1efc0f739e918075ed2197700759e027f821"} Oct 30 18:31:19.472403 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:19.471968 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:19.487125 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:19.486975 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:19.591530 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:19.591480 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" podStartSLOduration=7.211096704 podStartE2EDuration="21.591467833s" totalImagesPullingTime="14.380371129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:19.591345757 +0000 UTC m=+21.828414350" watchObservedRunningTime="2026-10-30 18:31:19.591467833 +0000 UTC m=+21.828536418" Oct 30 18:31:20.244425 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:20.244387 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:20.244425 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:20.244413 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:20.244660 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:20.244561 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:20.244660 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:20.244608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:20.244759 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:20.244682 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:20.244759 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:20.244728 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:20.474379 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:20.474352 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:20.474766 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:20.474437 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:20.491818 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:20.491787 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:21.451930 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:21.451270 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kvgwg"] Oct 30 18:31:21.451930 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:21.451398 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:21.451930 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:21.451501 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:21.453315 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:21.453289 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29d22"] Oct 30 18:31:21.453449 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:21.453388 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:21.453512 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:21.453484 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:21.465284 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:21.465253 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9vhcp"] Oct 30 18:31:21.465417 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:21.465345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:21.465455 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:21.465427 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:22.965970 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:22.965937 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kbnmf_7eb17ce4-c80f-476b-8682-c1a329bd82fb/dns-node-resolver/0.log" Oct 30 18:31:23.243718 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:23.243618 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:23.243718 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:23.243635 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:23.243980 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:23.243745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:23.243980 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:23.243742 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:23.243980 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:23.243837 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:23.243980 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:23.243920 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:24.106209 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:24.106181 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8j6q_422a1ecf-21d1-433f-8ede-d9de5a5940b7/node-ca/0.log" Oct 30 18:31:24.340108 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:24.340079 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:24.340289 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:24.340203 2570 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:24.340734 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:24.340715 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:31:25.243920 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:25.243886 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:25.243920 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:25.243908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:25.244458 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:25.243908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:25.244458 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:25.244020 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:25.244458 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:25.244077 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:25.244458 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:25.244130 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:27.243944 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:27.243850 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:27.243944 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:27.243876 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:27.244387 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:27.243868 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:27.244387 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:27.243992 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:27.244387 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:27.244052 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:27.244387 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:27.244134 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:29.243386 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:29.243357 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:29.243748 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:29.243358 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:29.243748 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:29.243463 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:29.243748 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:29.243358 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:29.243748 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:29.243541 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:29.243748 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:29.243629 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:31.243793 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:31.243755 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:31.243793 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:31.243778 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:31.244401 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:31.243786 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:31.244401 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.243880 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:31.244401 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.243943 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:31.244401 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.244007 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:31.980224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:31.980188 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:31.980443 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:31.980245 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:31.980443 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.980335 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:31.980443 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.980350 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:31.980443 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.980387 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs podName:7d565d10-f9d0-4450-a666-2c0805d53629 nodeName:}" failed. No retries permitted until 2026-10-30 18:32:03.980373887 +0000 UTC m=+66.217442459 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs") pod "network-metrics-daemon-29d22" (UID: "7d565d10-f9d0-4450-a666-2c0805d53629") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:31:31.980443 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:31.980411 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret podName:56d694ae-4469-45fe-b66d-36efcb67b5cb nodeName:}" failed. No retries permitted until 2026-10-30 18:32:03.980394509 +0000 UTC m=+66.217463096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret") pod "global-pull-secret-syncer-9vhcp" (UID: "56d694ae-4469-45fe-b66d-36efcb67b5cb") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:31:32.081481 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:32.081448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:32.081618 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:32.081568 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:31:32.081618 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:32.081581 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:31:32.081618 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:32.081590 2570 projected.go:196] Error preparing data for projected volume kube-api-access-snmxz for pod openshift-network-diagnostics/network-check-target-kvgwg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:32.081741 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:32.081635 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz podName:b028c48c-de12-42cd-b596-89cef2f6308f nodeName:}" failed. No retries permitted until 2026-10-30 18:32:04.081623734 +0000 UTC m=+66.318692306 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-snmxz" (UniqueName: "kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz") pod "network-check-target-kvgwg" (UID: "b028c48c-de12-42cd-b596-89cef2f6308f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:31:33.243837 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:33.243807 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:33.244263 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:33.243808 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:33.244263 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:33.243906 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:33.244263 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:33.244001 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:33.244263 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:33.243808 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:33.244263 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:33.244073 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:35.243380 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:35.243338 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:35.243828 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:35.243404 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:35.243828 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:35.243505 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:35.243828 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:35.243504 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:35.243828 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:35.243568 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:35.243828 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:35.243623 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:37.243379 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:37.243345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:37.243854 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:37.243345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:37.243854 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:37.243457 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:37.243854 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:37.243345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:37.243854 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:37.243540 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:37.243854 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:37.243621 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:39.244230 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:39.244196 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:39.244230 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:39.244221 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:39.244723 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:39.244351 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:39.244723 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:39.244388 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:39.244723 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:39.244450 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:39.244723 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:39.244555 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:41.244006 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.243974 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:41.244408 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.244018 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:41.244408 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.244018 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:41.244408 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:41.244102 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29d22" podUID="7d565d10-f9d0-4450-a666-2c0805d53629" Oct 30 18:31:41.244408 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:41.244195 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vhcp" podUID="56d694ae-4469-45fe-b66d-36efcb67b5cb" Oct 30 18:31:41.244408 ip-10-0-136-103 kubenswrapper[2570]: E1030 18:31:41.244281 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kvgwg" podUID="b028c48c-de12-42cd-b596-89cef2f6308f" Oct 30 18:31:41.556037 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.556015 2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeReady" Oct 30 18:31:41.556164 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.556120 2570 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:31:41.631680 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.631656 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mbxqk"] Oct 30 18:31:41.633223 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.633208 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.638695 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.638672 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tbzd6"] Oct 30 18:31:41.640360 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.640342 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.651688 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.651670 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:31:41.651793 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.651777 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-shvwg\"" Oct 30 18:31:41.652413 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.652398 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:31:41.652492 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.652462 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:31:41.655887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.655839 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/048e9269-1abc-446c-baed-e789c6e91df0-cert\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.656013 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.655908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqsbf\" (UniqueName: \"kubernetes.io/projected/048e9269-1abc-446c-baed-e789c6e91df0-kube-api-access-gqsbf\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.657082 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.657062 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mbxqk"] Oct 30 18:31:41.672101 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.672080 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sxvp2"] Oct 30 18:31:41.673605 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.673592 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.691042 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.691018 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tbzd6"] Oct 30 18:31:41.698260 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.698239 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sxvp2"] Oct 30 18:31:41.752193 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752152 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtqtd\"" Oct 30 18:31:41.752193 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752162 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:31:41.752193 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752188 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:31:41.752440 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752205 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:31:41.752440 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752219 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:31:41.752440 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752163 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:31:41.752440 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752192 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bz2z9\"" Oct 30 18:31:41.752440 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.752311 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:31:41.756458 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756436 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg97w\" (UniqueName: \"kubernetes.io/projected/a1a0af93-464e-4b58-bf1c-ab9de497715d-kube-api-access-dg97w\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.756593 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7fe0720-634c-4285-9b18-9a061f992421-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.756593 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1a0af93-464e-4b58-bf1c-ab9de497715d-metrics-tls\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.756701 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756627 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpfzx\" (UniqueName: \"kubernetes.io/projected/d7fe0720-634c-4285-9b18-9a061f992421-kube-api-access-hpfzx\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.756701 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756667 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7fe0720-634c-4285-9b18-9a061f992421-crio-socket\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.756780 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756706 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a0af93-464e-4b58-bf1c-ab9de497715d-config-volume\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.756780 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/048e9269-1abc-446c-baed-e789c6e91df0-cert\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.756860 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7fe0720-634c-4285-9b18-9a061f992421-data-volume\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.756860 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7fe0720-634c-4285-9b18-9a061f992421-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.756860 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a0af93-464e-4b58-bf1c-ab9de497715d-tmp-dir\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.757021 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.756904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqsbf\" (UniqueName: \"kubernetes.io/projected/048e9269-1abc-446c-baed-e789c6e91df0-kube-api-access-gqsbf\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.760394 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.760372 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/048e9269-1abc-446c-baed-e789c6e91df0-cert\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.819426 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.819356 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqsbf\" (UniqueName: \"kubernetes.io/projected/048e9269-1abc-446c-baed-e789c6e91df0-kube-api-access-gqsbf\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.857203 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857165 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1a0af93-464e-4b58-bf1c-ab9de497715d-metrics-tls\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.857386 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857226 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpfzx\" (UniqueName: \"kubernetes.io/projected/d7fe0720-634c-4285-9b18-9a061f992421-kube-api-access-hpfzx\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857386 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857258 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7fe0720-634c-4285-9b18-9a061f992421-crio-socket\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857386 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a0af93-464e-4b58-bf1c-ab9de497715d-config-volume\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.857589 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857435 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7fe0720-634c-4285-9b18-9a061f992421-data-volume\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857589 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857470 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7fe0720-634c-4285-9b18-9a061f992421-crio-socket\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857589 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7fe0720-634c-4285-9b18-9a061f992421-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857736 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a0af93-464e-4b58-bf1c-ab9de497715d-tmp-dir\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.857736 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg97w\" (UniqueName: \"kubernetes.io/projected/a1a0af93-464e-4b58-bf1c-ab9de497715d-kube-api-access-dg97w\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.857736 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7fe0720-634c-4285-9b18-9a061f992421-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857875 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857762 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7fe0720-634c-4285-9b18-9a061f992421-data-volume\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.857927 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.857896 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a0af93-464e-4b58-bf1c-ab9de497715d-tmp-dir\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.858132 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.858113 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7fe0720-634c-4285-9b18-9a061f992421-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.858132 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.858127 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a0af93-464e-4b58-bf1c-ab9de497715d-config-volume\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.859574 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.859548 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1a0af93-464e-4b58-bf1c-ab9de497715d-metrics-tls\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.859839 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.859820 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7fe0720-634c-4285-9b18-9a061f992421-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.887487 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.887458 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg97w\" (UniqueName: \"kubernetes.io/projected/a1a0af93-464e-4b58-bf1c-ab9de497715d-kube-api-access-dg97w\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:41.892285 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.892262 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpfzx\" (UniqueName: \"kubernetes.io/projected/d7fe0720-634c-4285-9b18-9a061f992421-kube-api-access-hpfzx\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.943206 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.943182 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:31:41.954520 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.954493 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:31:41.983364 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:41.983329 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:42.252948 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.252879 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mbxqk"] Oct 30 18:31:42.256471 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:42.256444 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048e9269_1abc_446c_baed_e789c6e91df0.slice/crio-74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331 WatchSource:0}: Error finding container 74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331: Status 404 returned error can't find the container with id 74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331 Oct 30 18:31:42.266637 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.266613 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tbzd6"] Oct 30 18:31:42.269995 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:42.269970 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7fe0720_634c_4285_9b18_9a061f992421.slice/crio-ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d WatchSource:0}: Error finding container ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d: Status 404 returned error can't find the container with id ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d Oct 30 18:31:42.291888 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.291861 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sxvp2"] Oct 30 18:31:42.295717 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:42.295690 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a0af93_464e_4b58_bf1c_ab9de497715d.slice/crio-610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f WatchSource:0}: Error finding container 610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f: Status 404 returned error can't find the container with id 610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f Oct 30 18:31:42.522062 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.521966 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tbzd6" event={"ID":"d7fe0720-634c-4285-9b18-9a061f992421","Type":"ContainerStarted","Data":"ccf77b1ee0d2590d7db26f7c819429dfb6732edaa22332223e33832c7b4acf6f"} Oct 30 18:31:42.522062 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.522034 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tbzd6" event={"ID":"d7fe0720-634c-4285-9b18-9a061f992421","Type":"ContainerStarted","Data":"ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d"} Oct 30 18:31:42.523096 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.523072 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mbxqk" event={"ID":"048e9269-1abc-446c-baed-e789c6e91df0","Type":"ContainerStarted","Data":"74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331"} Oct 30 18:31:42.524118 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:42.524097 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sxvp2" event={"ID":"a1a0af93-464e-4b58-bf1c-ab9de497715d","Type":"ContainerStarted","Data":"610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f"} Oct 30 18:31:43.243605 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.243526 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:31:43.243941 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.243926 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:31:43.244597 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.244573 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:31:43.264292 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.264238 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:31:43.264637 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.264349 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:31:43.264637 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.264390 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:31:43.264637 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.264602 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:31:43.279218 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.279192 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:31:43.279376 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.279193 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:31:43.279376 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.279278 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:31:43.529725 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:43.529605 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tbzd6" event={"ID":"d7fe0720-634c-4285-9b18-9a061f992421","Type":"ContainerStarted","Data":"66b8610563ddf6053e8c614f47b17c716c69015fa465e27fec589b4bdfe52f44"} Oct 30 18:31:44.194415 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.194351 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-rk24f"] Oct 30 18:31:44.197772 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.196674 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.216306 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.216266 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:31:44.233567 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.231465 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 18:31:44.233567 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.233287 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-rk24f"] Oct 30 18:31:44.233567 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.233461 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-f9b88\"" Oct 30 18:31:44.233567 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.233539 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 18:31:44.275273 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.275218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.275712 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.275297 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.275712 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.275419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb83f0f-6234-4239-8065-df0c41d41b32-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.275712 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.275446 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6pmr\" (UniqueName: \"kubernetes.io/projected/1eb83f0f-6234-4239-8065-df0c41d41b32-kube-api-access-x6pmr\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.316409 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.316367 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:31:44.316671 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.316419 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:31:44.375876 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.375788 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.375876 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.375829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.375876 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.375879 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb83f0f-6234-4239-8065-df0c41d41b32-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.376130 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.375900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6pmr\" (UniqueName: \"kubernetes.io/projected/1eb83f0f-6234-4239-8065-df0c41d41b32-kube-api-access-x6pmr\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.377329 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.377275 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb83f0f-6234-4239-8065-df0c41d41b32-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.378785 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.378751 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.380223 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.380182 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.411937 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.411903 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6pmr\" (UniqueName: \"kubernetes.io/projected/1eb83f0f-6234-4239-8065-df0c41d41b32-kube-api-access-x6pmr\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.526843 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.526802 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:31:44.533248 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.533216 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sxvp2" event={"ID":"a1a0af93-464e-4b58-bf1c-ab9de497715d","Type":"ContainerStarted","Data":"08b3118ca04b6f48d6bf0768cb24c0d9389d9c888ee81c22abb24308da521b7a"} Oct 30 18:31:44.533342 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.533286 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sxvp2" event={"ID":"a1a0af93-464e-4b58-bf1c-ab9de497715d","Type":"ContainerStarted","Data":"a8f8dd2bed998ce9ee3aef33e1d092026783294bdcd0daaff336b39d76a06c35"} Oct 30 18:31:44.533385 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.533344 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:44.593271 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.593214 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-sxvp2" podStartSLOduration=1.768469177 podStartE2EDuration="3.593193739s" totalImagesPullingTime="1.824724562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:44.593016209 +0000 UTC m=+46.830084806" watchObservedRunningTime="2026-10-30 18:31:44.593193739 +0000 UTC m=+46.830262335" Oct 30 18:31:44.831609 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:44.831573 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-rk24f"] Oct 30 18:31:44.835353 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:44.835327 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb83f0f_6234_4239_8065_df0c41d41b32.slice/crio-a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e WatchSource:0}: Error finding container a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e: Status 404 returned error can't find the container with id a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e Oct 30 18:31:45.538222 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.538187 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tbzd6" event={"ID":"d7fe0720-634c-4285-9b18-9a061f992421","Type":"ContainerStarted","Data":"0b6aad159f4dcb9746c86e4d7ddef70349f3cc10704e033f643c64db63235ead"} Oct 30 18:31:45.539765 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.539737 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mbxqk" event={"ID":"048e9269-1abc-446c-baed-e789c6e91df0","Type":"ContainerStarted","Data":"a1a3996130db351e83212b04b3a21f97477815f41595ee5436257c3d7504f855"} Oct 30 18:31:45.541074 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.541045 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" event={"ID":"1eb83f0f-6234-4239-8065-df0c41d41b32","Type":"ContainerStarted","Data":"a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e"} Oct 30 18:31:45.563087 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.563022 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sxvp2_a1a0af93-464e-4b58-bf1c-ab9de497715d/dns/0.log" Oct 30 18:31:45.587377 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.587328 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tbzd6" podStartSLOduration=2.528873099 podStartE2EDuration="4.587310052s" totalImagesPullingTime="2.058436953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:45.586875676 +0000 UTC m=+47.823944271" watchObservedRunningTime="2026-10-30 18:31:45.587310052 +0000 UTC m=+47.824378647" Oct 30 18:31:45.633029 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.632979 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mbxqk" podStartSLOduration=1.823868131 podStartE2EDuration="4.632945373s" totalImagesPullingTime="2.809077242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:45.632407963 +0000 UTC m=+47.869476558" watchObservedRunningTime="2026-10-30 18:31:45.632945373 +0000 UTC m=+47.870013966" Oct 30 18:31:45.763484 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:45.763458 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sxvp2_a1a0af93-464e-4b58-bf1c-ab9de497715d/kube-rbac-proxy/0.log" Oct 30 18:31:46.162366 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:46.162331 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kbnmf_7eb17ce4-c80f-476b-8682-c1a329bd82fb/dns-node-resolver/0.log" Oct 30 18:31:46.545724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:46.545686 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" event={"ID":"1eb83f0f-6234-4239-8065-df0c41d41b32","Type":"ContainerStarted","Data":"9cfbc7aafbc4920b2c7db43cc1ca334a6f91f287b20652f7d077f74af8c3019c"} Oct 30 18:31:46.546269 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:46.545739 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" event={"ID":"1eb83f0f-6234-4239-8065-df0c41d41b32","Type":"ContainerStarted","Data":"de06d37e9a621b2f6f51d39e8ef45e59baee628e172f862e8c6c8e69ca3a810e"} Oct 30 18:31:46.592710 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:46.592657 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" podStartSLOduration=1.504950214 podStartE2EDuration="2.592645707s" totalImagesPullingTime="1.087695493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:46.592155061 +0000 UTC m=+48.829223719" watchObservedRunningTime="2026-10-30 18:31:46.592645707 +0000 UTC m=+48.829714300" Oct 30 18:31:47.567046 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:47.567019 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8j6q_422a1ecf-21d1-433f-8ede-d9de5a5940b7/node-ca/0.log" Oct 30 18:31:48.366260 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:48.366232 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mbxqk_048e9269-1abc-446c-baed-e789c6e91df0/serve-healthcheck-canary/0.log" Oct 30 18:31:48.987642 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:48.987608 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x"] Oct 30 18:31:48.989664 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:48.989648 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.018817 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.018773 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.018817 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.018823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.019067 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.018922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.019067 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.018964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.019067 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.018991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.019172 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.019063 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6lh6\" (UniqueName: \"kubernetes.io/projected/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-api-access-k6lh6\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.039340 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.039308 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x"] Oct 30 18:31:49.049132 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.049110 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-89zdx\"" Oct 30 18:31:49.049328 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.049311 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:31:49.049429 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.049416 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 18:31:49.053089 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.053072 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 18:31:49.064499 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.064476 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s76gv"] Oct 30 18:31:49.066613 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.066596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.119659 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.119626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.119849 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.119668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6lh6\" (UniqueName: \"kubernetes.io/projected/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-api-access-k6lh6\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.119849 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.119698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.119849 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.119830 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120051 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.119903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.120051 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.119943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-tls\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120051 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-textfile\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120185 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.120185 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120107 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-sys\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120185 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120108 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.120326 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120326 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120259 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-wtmp\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120326 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-metrics-client-ca\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120478 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-root\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120478 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120410 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxc2c\" (UniqueName: \"kubernetes.io/projected/bbe9260b-f403-4983-8e9a-6c328064b4c1-kube-api-access-zxc2c\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.120478 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.120724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120692 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.120895 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.120875 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.122321 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.122299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.122715 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.122692 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.143470 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.143442 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:31:49.144038 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.144020 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:31:49.144182 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.144164 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twcvb\"" Oct 30 18:31:49.144254 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.144199 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:31:49.145158 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.145140 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6lh6\" (UniqueName: \"kubernetes.io/projected/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-api-access-k6lh6\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.221042 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221008 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-root\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221042 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxc2c\" (UniqueName: \"kubernetes.io/projected/bbe9260b-f403-4983-8e9a-6c328064b4c1-kube-api-access-zxc2c\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221084 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221113 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-tls\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221121 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-root\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221174 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-textfile\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221224 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221210 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-sys\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221461 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221238 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221461 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-wtmp\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221461 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221288 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-metrics-client-ca\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221461 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221313 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-sys\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221461 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221440 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-wtmp\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221705 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221654 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221761 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221718 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-textfile\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.221807 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.221776 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-metrics-client-ca\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.223374 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.223354 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-tls\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.223505 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.223485 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.250173 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.250119 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxc2c\" (UniqueName: \"kubernetes.io/projected/bbe9260b-f403-4983-8e9a-6c328064b4c1-kube-api-access-zxc2c\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.301274 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.301227 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:31:49.378275 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.378247 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:31:49.387257 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:49.387230 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe9260b_f403_4983_8e9a_6c328064b4c1.slice/crio-93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569 WatchSource:0}: Error finding container 93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569: Status 404 returned error can't find the container with id 93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569 Oct 30 18:31:49.555477 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.555384 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s76gv" event={"ID":"bbe9260b-f403-4983-8e9a-6c328064b4c1","Type":"ContainerStarted","Data":"93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569"} Oct 30 18:31:49.607351 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:49.607320 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x"] Oct 30 18:31:49.610458 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:49.610435 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513b3b1c_2fd2_42df_9438_4f7e52bf43bc.slice/crio-9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01 WatchSource:0}: Error finding container 9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01: Status 404 returned error can't find the container with id 9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01 Oct 30 18:31:50.292277 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.292236 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:31:50.295797 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.295772 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.323405 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323371 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 18:31:50.323549 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323437 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 18:31:50.323549 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323460 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 18:31:50.323549 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323477 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 18:31:50.323549 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323516 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 18:31:50.323549 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323532 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 18:31:50.323549 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323535 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 18:31:50.323795 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323567 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 18:31:50.323795 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.323588 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tnxh8\"" Oct 30 18:31:50.330299 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330337 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330453 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330568 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330613 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330682 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.330724 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.330705 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.346288 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.346262 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 18:31:50.353698 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.353668 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:31:50.431610 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431574 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431772 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431772 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431683 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431772 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431772 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431728 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431772 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431757 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431938 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431782 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431938 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431813 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431938 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431938 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.431938 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431929 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.432173 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.431999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.432173 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.432035 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.438410 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.438370 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.439103 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.439070 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.439887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.439673 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.440783 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.440710 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.442813 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.442786 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.442912 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.442883 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.443969 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.443299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.443969 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.443528 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.443969 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.443593 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.445690 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.445654 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.446035 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.446004 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.448169 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.448144 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.458090 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.458066 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.561178 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.561097 2570 generic.go:356] "Generic (PLEG): container finished" podID="bbe9260b-f403-4983-8e9a-6c328064b4c1" containerID="66dac793a90c45fa3fe5e3a86946e4a83feff2765cf0174ee18e2978cf2de266" exitCode=0 Oct 30 18:31:50.561327 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.561191 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s76gv" event={"ID":"bbe9260b-f403-4983-8e9a-6c328064b4c1","Type":"ContainerDied","Data":"66dac793a90c45fa3fe5e3a86946e4a83feff2765cf0174ee18e2978cf2de266"} Oct 30 18:31:50.562422 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.562392 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" event={"ID":"513b3b1c-2fd2-42df-9438-4f7e52bf43bc","Type":"ContainerStarted","Data":"9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01"} Oct 30 18:31:50.615404 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.615372 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:31:50.957309 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:50.957278 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:31:50.977106 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:50.977078 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbf3cae_5e36_4c89_adf9_32f652e9b89f.slice/crio-0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b WatchSource:0}: Error finding container 0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b: Status 404 returned error can't find the container with id 0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b Oct 30 18:31:51.567133 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.567099 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" event={"ID":"513b3b1c-2fd2-42df-9438-4f7e52bf43bc","Type":"ContainerStarted","Data":"1940c6d1f0dfe35ebf9b311ecd7b16017833e19dc26631d9d441c3b428e32bc6"} Oct 30 18:31:51.567589 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.567161 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" event={"ID":"513b3b1c-2fd2-42df-9438-4f7e52bf43bc","Type":"ContainerStarted","Data":"db4f30f56a4c017ddccd9e2eca5ef227d486448e6418b5f539b5c5f4a4f53f55"} Oct 30 18:31:51.567589 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.567186 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" event={"ID":"513b3b1c-2fd2-42df-9438-4f7e52bf43bc","Type":"ContainerStarted","Data":"4d55e5cce08a4c6800fd49aab42e38c28b6ca95f482f5f50f0c7824406e461c0"} Oct 30 18:31:51.568442 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.568415 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b"} Oct 30 18:31:51.570381 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.570359 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s76gv" event={"ID":"bbe9260b-f403-4983-8e9a-6c328064b4c1","Type":"ContainerStarted","Data":"8f97142846a733b829b6cd49c066ec80c9eb66bc68d96a3bc1bb697a83807c7d"} Oct 30 18:31:51.570466 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.570401 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s76gv" event={"ID":"bbe9260b-f403-4983-8e9a-6c328064b4c1","Type":"ContainerStarted","Data":"5b6d7e2d178a48054bc22ebeda8be71526baf8794a06273f9807e49311697a89"} Oct 30 18:31:51.617871 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.617807 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" podStartSLOduration=2.346457653 podStartE2EDuration="3.617788938s" totalImagesPullingTime="1.271331285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:51.616972688 +0000 UTC m=+53.854041274" watchObservedRunningTime="2026-10-30 18:31:51.617788938 +0000 UTC m=+53.854857534" Oct 30 18:31:51.668320 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:51.668256 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s76gv" podStartSLOduration=1.6254761210000002 podStartE2EDuration="2.668240943s" totalImagesPullingTime="957.928992ms" totalInitContainerRuntime="84.83583ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:51.667477817 +0000 UTC m=+53.904546412" watchObservedRunningTime="2026-10-30 18:31:51.668240943 +0000 UTC m=+53.905309537" Oct 30 18:31:52.489699 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:52.489624 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:31:52.580330 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:52.580294 2570 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf" exitCode=0 Oct 30 18:31:52.580810 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:52.580384 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf"} Oct 30 18:31:53.652754 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.652714 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cft4b"] Oct 30 18:31:53.655057 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.655031 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.673740 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.673712 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cft4b"] Oct 30 18:31:53.686627 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.686595 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 30 18:31:53.686627 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.686626 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rx4vd\"" Oct 30 18:31:53.762125 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.762092 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9shbt\" (UniqueName: \"kubernetes.io/projected/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-kube-api-access-9shbt\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.762261 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.762142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.862877 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.862837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9shbt\" (UniqueName: \"kubernetes.io/projected/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-kube-api-access-9shbt\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.863069 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.862900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.865819 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.865791 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.889076 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.889043 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9shbt\" (UniqueName: \"kubernetes.io/projected/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-kube-api-access-9shbt\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:53.968055 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:53.967977 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:54.222866 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.222837 2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55f45f79df-6q4pk"] Oct 30 18:31:54.225764 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.225741 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.250806 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.250785 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 30 18:31:54.266628 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.266774 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266641 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-federate-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.266774 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266670 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-metrics-client-ca\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.266887 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.266941 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266899 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6b7\" (UniqueName: \"kubernetes.io/projected/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-kube-api-access-qt6b7\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.266941 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266930 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.267078 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.266969 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.267078 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.267012 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-serving-certs-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.279416 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.279392 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 30 18:31:54.279551 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.279392 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 30 18:31:54.279551 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.279392 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 30 18:31:54.279722 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.279430 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-d2md9\"" Oct 30 18:31:54.279722 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.279442 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 30 18:31:54.279722 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.279573 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 30 18:31:54.280763 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.280698 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55f45f79df-6q4pk"] Oct 30 18:31:54.292364 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.292336 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cft4b"] Oct 30 18:31:54.293666 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:54.293641 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8123ce_1ece_4d21_8bcb_4c7effd34d1d.slice/crio-e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04 WatchSource:0}: Error finding container e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04: Status 404 returned error can't find the container with id e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04 Oct 30 18:31:54.368159 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368130 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368159 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368162 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-federate-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368325 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-metrics-client-ca\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368325 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368199 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368460 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6b7\" (UniqueName: \"kubernetes.io/projected/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-kube-api-access-qt6b7\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368460 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368460 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.368460 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.368455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-serving-certs-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.369062 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.369040 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-metrics-client-ca\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.369194 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.369170 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-serving-certs-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.369647 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.369600 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.371121 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.371102 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.371328 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.371307 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-federate-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.371796 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.371775 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.372050 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.372024 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.394539 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.394514 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6b7\" (UniqueName: \"kubernetes.io/projected/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-kube-api-access-qt6b7\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.543385 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.543351 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sxvp2" Oct 30 18:31:54.548774 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.548752 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:31:54.600143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.598391 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c"} Oct 30 18:31:54.600143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.598481 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b"} Oct 30 18:31:54.600143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.598521 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156"} Oct 30 18:31:54.600143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.598547 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7"} Oct 30 18:31:54.600143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.598581 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb"} Oct 30 18:31:54.600817 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.600782 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" event={"ID":"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d","Type":"ContainerStarted","Data":"e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04"} Oct 30 18:31:54.868092 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:31:54.868056 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68a11ea_7b0c_414c_a1dd_a39b3b89472e.slice/crio-bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce WatchSource:0}: Error finding container bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce: Status 404 returned error can't find the container with id bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce Oct 30 18:31:54.868854 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:54.868732 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55f45f79df-6q4pk"] Oct 30 18:31:55.609258 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:55.609218 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerStarted","Data":"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a"} Oct 30 18:31:55.611268 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:55.611228 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" event={"ID":"c68a11ea-7b0c-414c-a1dd-a39b3b89472e","Type":"ContainerStarted","Data":"bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce"} Oct 30 18:31:55.683172 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:55.683112 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.30502041 podStartE2EDuration="5.683093488s" totalImagesPullingTime="3.366840338s" totalInitContainerRuntime="11.23274ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:55.682343444 +0000 UTC m=+57.919412039" watchObservedRunningTime="2026-10-30 18:31:55.683093488 +0000 UTC m=+57.920162083" Oct 30 18:31:56.615918 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:56.615883 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" event={"ID":"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d","Type":"ContainerStarted","Data":"b0d2fa015ca3b8b1ef072fc136101cda53db32f2edfdb8335be6e7bd9f355f02"} Oct 30 18:31:56.616385 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:56.616362 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:56.621143 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:56.621121 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:31:56.662121 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:56.662072 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" podStartSLOduration=2.16997713 podStartE2EDuration="3.662058657s" totalImagesPullingTime="1.492081527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:56.661121155 +0000 UTC m=+58.898189749" watchObservedRunningTime="2026-10-30 18:31:56.662058657 +0000 UTC m=+58.899127250" Oct 30 18:31:57.621125 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:57.621092 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" event={"ID":"c68a11ea-7b0c-414c-a1dd-a39b3b89472e","Type":"ContainerStarted","Data":"50af6cedb7bae7394c6a860b0d60adb036f949c44131d7d04395293bc59aaf01"} Oct 30 18:31:57.621521 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:57.621154 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" event={"ID":"c68a11ea-7b0c-414c-a1dd-a39b3b89472e","Type":"ContainerStarted","Data":"0a105fe370c65bbf4d14eb244ba1698078d0ce124af0acd3bfb9211c45141363"} Oct 30 18:31:57.621521 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:57.621174 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" event={"ID":"c68a11ea-7b0c-414c-a1dd-a39b3b89472e","Type":"ContainerStarted","Data":"d18e43f27e7b13f0e3b7675bd1d5d0ad50f71bea9681b0904fb9a1dde52e952f"} Oct 30 18:31:57.705382 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:31:57.705332 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" podStartSLOduration=2.007006834 podStartE2EDuration="3.705319676s" totalImagesPullingTime="1.698312842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:57.704624987 +0000 UTC m=+59.941693580" watchObservedRunningTime="2026-10-30 18:31:57.705319676 +0000 UTC m=+59.942388270" Oct 30 18:32:04.066732 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.066695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:04.067247 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.066747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:04.085389 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.085336 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:32:04.090035 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.090012 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:04.098414 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.098391 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:32:04.109460 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.109431 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:04.168444 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.167900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:04.184591 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.184566 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:32:04.204882 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.204849 2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:32:04.221168 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.221138 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:04.276377 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.276341 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:32:04.280593 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.280573 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:04.284999 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.284930 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:32:04.288991 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.288968 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:04.294270 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.294245 2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:32:04.298321 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.298284 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:04.604382 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.604314 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29d22"] Oct 30 18:32:04.607239 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:32:04.607213 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d565d10_f9d0_4450_a666_2c0805d53629.slice/crio-eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a WatchSource:0}: Error finding container eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a: Status 404 returned error can't find the container with id eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a Oct 30 18:32:04.636353 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.636332 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kvgwg"] Oct 30 18:32:04.638196 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:32:04.638171 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb028c48c_de12_42cd_b596_89cef2f6308f.slice/crio-dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840 WatchSource:0}: Error finding container dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840: Status 404 returned error can't find the container with id dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840 Oct 30 18:32:04.645616 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.645589 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kvgwg" event={"ID":"b028c48c-de12-42cd-b596-89cef2f6308f","Type":"ContainerStarted","Data":"dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840"} Oct 30 18:32:04.646673 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.646654 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29d22" event={"ID":"7d565d10-f9d0-4450-a666-2c0805d53629","Type":"ContainerStarted","Data":"eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a"} Oct 30 18:32:04.666908 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:04.666866 2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9vhcp"] Oct 30 18:32:04.668832 ip-10-0-136-103 kubenswrapper[2570]: W1030 18:32:04.668808 2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56d694ae_4469_45fe_b66d_36efcb67b5cb.slice/crio-22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19 WatchSource:0}: Error finding container 22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19: Status 404 returned error can't find the container with id 22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19 Oct 30 18:32:05.652152 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:05.652110 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9vhcp" event={"ID":"56d694ae-4469-45fe-b66d-36efcb67b5cb","Type":"ContainerStarted","Data":"22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19"} Oct 30 18:32:06.658931 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:06.658885 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29d22" event={"ID":"7d565d10-f9d0-4450-a666-2c0805d53629","Type":"ContainerStarted","Data":"2fabc0ab1c8de4e49cf1d5b9070ff45a10486158bf560d1f6672441202b112f1"} Oct 30 18:32:06.659384 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:06.658948 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29d22" event={"ID":"7d565d10-f9d0-4450-a666-2c0805d53629","Type":"ContainerStarted","Data":"d41e3eb2917ec71ab3d64ee65983d38135e7acf7f61ffcfb7120b3a26ae8c9ce"} Oct 30 18:32:06.725943 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:06.725885 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29d22" podStartSLOduration=67.676978153 podStartE2EDuration="1m8.725866435s" totalImagesPullingTime="1.048888282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:06.71025804 +0000 UTC m=+68.947326626" watchObservedRunningTime="2026-10-30 18:32:06.725866435 +0000 UTC m=+68.962935080" Oct 30 18:32:08.667759 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:08.667723 2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kvgwg" event={"ID":"b028c48c-de12-42cd-b596-89cef2f6308f","Type":"ContainerStarted","Data":"8b69aa68352e78049f150c88dc0be1a7aa06ac30b7170035fb94c61378332075"} Oct 30 18:32:08.668228 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:08.667874 2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:08.716990 ip-10-0-136-103 kubenswrapper[2570]: I1030 18:32:08.716916 2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kvgwg" podStartSLOduration=67.607602842 podStartE2EDuration="1m10.716902301s" totalImagesPullingTime="3.109299459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:08.716718472 +0000 UTC m=+70.953787067" watchObservedRunningTime="2026-10-30 18:32:08.716902301 +0000 UTC m=+70.953970933" Oct 30 18:32:09.399535 ip-10-0-136-103 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 18:32:09.416358 ip-10-0-136-103 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 18:32:09.416531 ip-10-0-136-103 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 18:32:09.416582 ip-10-0-136-103 systemd[1]: kubelet.service: Consumed 3.943s CPU time. Oct 30 18:32:09.430626 ip-10-0-136-103 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:32:09.492098 ip-10-0-136-103 kubenswrapper[6160]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:32:09.492098 ip-10-0-136-103 kubenswrapper[6160]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:32:09.492098 ip-10-0-136-103 kubenswrapper[6160]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:32:09.492098 ip-10-0-136-103 kubenswrapper[6160]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:32:09.494151 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494066 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.494151 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494150 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.494151 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494154 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494158 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494161 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494164 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494171 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494174 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494177 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494186 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494189 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494192 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494195 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494198 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494200 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494203 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494205 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494208 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494211 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494213 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494216 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.494246 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494219 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494222 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494224 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494227 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494230 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494232 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494235 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494237 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494243 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494246 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494249 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494252 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494255 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494258 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494260 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494263 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494266 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494268 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494271 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494273 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.494725 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494276 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494284 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494287 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494290 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494292 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494295 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494297 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494300 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494302 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494305 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494308 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494310 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494313 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494316 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494318 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494321 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494324 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494326 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494329 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.495250 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494333 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494335 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494338 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494341 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494344 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494346 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494349 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494351 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494354 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494356 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494359 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494362 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494364 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494367 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494369 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494378 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494380 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494383 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494385 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.495744 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494388 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494391 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494393 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494396 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494398 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494401 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494404 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494406 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494410 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494415 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494418 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494421 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494424 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494427 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494430 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494432 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494436 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494439 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494441 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.496238 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494444 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494446 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494449 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494454 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494458 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494461 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494464 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.494467 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495276 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495283 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495287 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495290 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495294 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495305 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495308 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495311 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495314 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495317 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495320 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495323 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.496734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495326 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495329 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495331 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495334 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495337 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495340 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495342 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495345 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495348 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495350 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495353 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495356 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495358 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495361 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495363 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495368 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495372 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495375 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495378 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.497249 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495381 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495384 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495386 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495389 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495392 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495395 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495398 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495401 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495404 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495406 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495409 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495412 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495415 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495419 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495422 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495424 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495427 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495430 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495433 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495435 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.497758 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495438 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495441 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495443 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495446 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495448 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495451 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495453 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495456 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495458 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495461 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495463 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495466 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495468 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495471 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495474 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495476 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495479 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495483 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495485 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495488 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.498259 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495491 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495494 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495496 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495499 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495501 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495503 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495506 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495508 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495511 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495513 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495516 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495519 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495521 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495524 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495526 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495529 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495532 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495535 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495537 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.498754 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495540 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495542 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495545 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495548 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495550 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495552 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495555 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495558 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495560 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495563 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495570 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495577 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495579 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495582 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495585 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.495587 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496207 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496216 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496219 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.499265 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496222 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496226 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496229 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496232 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496235 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496237 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496240 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496243 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496247 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496250 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496253 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496256 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496259 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496261 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496264 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496266 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496269 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496272 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496274 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.499734 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496277 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496279 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496282 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496285 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496288 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496293 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496296 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496299 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496302 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496304 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496307 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496310 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496313 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496315 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496318 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496320 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496323 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496325 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496328 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.500325 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496331 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496333 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496335 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496338 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496341 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496343 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496346 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496349 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496351 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496354 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496356 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496359 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496361 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496364 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496366 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496369 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496374 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496377 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496381 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496385 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.501086 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496387 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496390 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496393 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496395 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496398 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496400 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496403 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496405 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496408 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496410 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496413 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496416 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496418 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496421 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496424 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496428 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496431 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496434 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496436 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.501625 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496439 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496442 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496444 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496447 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496449 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496452 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496455 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496457 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496460 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496463 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496465 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496469 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496472 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496475 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496478 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496480 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496483 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496486 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496489 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496491 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.502111 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496494 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496496 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496499 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496501 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496504 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.496507 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496605 6160 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496622 6160 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496631 6160 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496636 6160 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496640 6160 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496644 6160 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496648 6160 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496652 6160 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496656 6160 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496659 6160 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496662 6160 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496666 6160 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496669 6160 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496672 6160 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496675 6160 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496678 6160 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496681 6160 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:32:09.502599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496685 6160 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496689 6160 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496692 6160 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496696 6160 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496700 6160 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496703 6160 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496707 6160 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496710 6160 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496713 6160 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496717 6160 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496720 6160 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496723 6160 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496743 6160 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496747 6160 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496751 6160 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496756 6160 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496759 6160 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496762 6160 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496765 6160 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496768 6160 flags.go:64] FLAG: --enable-server="true" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496772 6160 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496790 6160 flags.go:64] FLAG: --event-burst="100" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496794 6160 flags.go:64] FLAG: --event-qps="50" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496797 6160 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:32:09.503179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496801 6160 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496804 6160 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496809 6160 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496812 6160 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496815 6160 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496818 6160 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496822 6160 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496825 6160 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496829 6160 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496833 6160 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496837 6160 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496840 6160 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496843 6160 flags.go:64] FLAG: --feature-gates="" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496847 6160 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496851 6160 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496854 6160 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496857 6160 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496860 6160 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496863 6160 flags.go:64] FLAG: --help="false" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496866 6160 flags.go:64] FLAG: --hostname-override="ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496871 6160 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496874 6160 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496877 6160 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496881 6160 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:32:09.503778 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496884 6160 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496887 6160 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496890 6160 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496893 6160 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496895 6160 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496899 6160 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496902 6160 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496905 6160 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496908 6160 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496910 6160 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496914 6160 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496917 6160 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496919 6160 flags.go:64] FLAG: --lock-file="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496922 6160 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496925 6160 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496928 6160 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496935 6160 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496938 6160 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496941 6160 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496944 6160 flags.go:64] FLAG: --logging-format="text" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496947 6160 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496963 6160 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496967 6160 flags.go:64] FLAG: --manifest-url="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496970 6160 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496975 6160 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:32:09.504430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496978 6160 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496982 6160 flags.go:64] FLAG: --max-pods="110" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496985 6160 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496989 6160 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496992 6160 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496995 6160 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.496998 6160 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497001 6160 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497004 6160 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497012 6160 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497015 6160 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497018 6160 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497021 6160 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497024 6160 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497027 6160 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497030 6160 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497033 6160 flags.go:64] FLAG: --port="10250" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497036 6160 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497039 6160 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09e70bc2ab84269f0" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497042 6160 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497045 6160 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497049 6160 flags.go:64] FLAG: --register-node="true" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497052 6160 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497058 6160 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497061 6160 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497063 6160 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:32:09.505132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497066 6160 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497070 6160 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497073 6160 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497077 6160 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497080 6160 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497083 6160 flags.go:64] FLAG: --runonce="false" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497086 6160 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497089 6160 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497092 6160 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497096 6160 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497099 6160 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497102 6160 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497105 6160 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497108 6160 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497111 6160 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497114 6160 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497117 6160 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497119 6160 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497122 6160 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497125 6160 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497128 6160 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497134 6160 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497137 6160 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497140 6160 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497143 6160 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:32:09.505771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497146 6160 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497149 6160 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497151 6160 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497154 6160 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497159 6160 flags.go:64] FLAG: --v="2" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497163 6160 flags.go:64] FLAG: --version="false" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497167 6160 flags.go:64] FLAG: --vmodule="" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497171 6160 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497175 6160 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497314 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497318 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497322 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497325 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497327 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497330 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497332 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497336 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497339 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497342 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497345 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497349 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.506386 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497352 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497355 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497358 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497361 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497363 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497366 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497369 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497371 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497374 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497376 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497379 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497382 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497385 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497387 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497390 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497393 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497396 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497398 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497401 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497404 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.506908 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497407 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497410 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497412 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497415 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497418 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497421 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497423 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497427 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497430 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497432 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497435 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497437 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497440 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497443 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497445 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497448 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497451 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497454 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497459 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.507634 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497462 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497465 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497468 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497470 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497473 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497476 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497478 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497481 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497483 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497486 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497489 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497491 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497493 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497496 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497499 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497501 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497504 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497510 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497513 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497515 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.508152 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497522 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497525 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497527 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497530 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497532 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497535 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497538 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497540 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497543 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497545 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497548 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497550 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497553 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497555 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497558 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497561 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497564 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497566 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497569 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.508756 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497571 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497574 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497576 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497579 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497581 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497584 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497586 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497589 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497591 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497594 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497597 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497600 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497602 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497606 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497608 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497611 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.509411 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.497637 6160 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497672 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497677 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497680 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497682 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497685 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497688 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497690 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497693 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497695 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497698 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497701 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497704 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497706 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497709 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497711 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497714 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497717 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497719 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497722 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.509815 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497724 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497727 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497729 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497732 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497734 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497737 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497739 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497742 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497746 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497752 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497756 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497759 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497762 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497765 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497768 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497771 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497773 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497776 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497778 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.510327 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497781 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497783 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497786 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497788 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497791 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497795 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497798 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497801 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497804 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497807 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497810 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497812 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497815 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497817 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497820 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497823 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497825 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497828 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497830 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497833 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.510826 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497835 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497838 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497842 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497845 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497848 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497851 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497853 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497857 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497861 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497865 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497869 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497872 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497874 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497877 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497880 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497882 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497885 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497889 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497892 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.511394 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497894 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497897 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497900 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497902 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497905 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497908 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497910 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497913 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497915 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497918 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497920 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497923 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497925 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497928 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497930 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497934 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497937 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497940 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497943 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497945 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.511899 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497948 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497962 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497965 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497967 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497970 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497973 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497975 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497978 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.497980 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.502818 6160 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.502835 6160 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.502858 6160 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.502863 6160 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502927 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502933 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502937 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502940 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502943 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502946 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502963 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.512400 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502968 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502972 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502976 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502978 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502981 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502984 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502987 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502990 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502992 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502995 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.502998 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503000 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503003 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503006 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503008 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503012 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503017 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503020 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503023 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503026 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.512883 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503029 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503032 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503035 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503038 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503040 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503043 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503046 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503048 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503051 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503054 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503056 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503059 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503062 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503065 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503067 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503070 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503073 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503075 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503078 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503081 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.513388 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503084 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503087 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503090 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503093 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503095 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503098 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503100 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503103 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503107 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503110 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503114 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503116 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503119 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503122 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503125 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503127 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503130 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503133 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503135 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.513884 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503138 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503141 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503143 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503146 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503148 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503151 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503154 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503157 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503159 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503162 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503165 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503167 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503170 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503173 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503176 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503178 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503181 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503184 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503187 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.514384 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503189 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503192 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503195 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503197 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503200 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503203 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503205 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503208 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503211 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503213 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503216 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503219 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503221 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503224 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503227 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503229 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503232 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503235 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503238 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.514871 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503240 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503243 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.503269 6160 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503423 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503429 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503432 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503434 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503438 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503441 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503444 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503446 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503449 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503452 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503454 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503457 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503459 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.515358 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503462 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503465 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503467 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503470 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503473 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503475 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503478 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503480 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503483 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503486 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503488 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503491 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503493 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503497 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503501 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503504 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503506 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503509 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503512 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.515736 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503514 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503518 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503522 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503524 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503527 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503530 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503533 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503536 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503538 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503541 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503543 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503546 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503548 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503551 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503554 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503556 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503559 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503562 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503565 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.516268 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503567 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503570 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503572 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503575 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503578 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503580 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503583 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503585 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503588 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503591 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503593 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503596 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503598 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503601 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503603 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503606 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503608 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503610 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503613 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503616 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.516757 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503619 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503622 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503625 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503628 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503630 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503633 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503636 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503639 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503641 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503644 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503646 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503649 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503651 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503654 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503656 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503659 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503661 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503664 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503666 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.517409 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503669 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503671 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503674 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503677 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503679 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503682 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503684 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503687 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503689 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503692 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503694 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503697 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503699 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503702 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503705 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503708 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.518124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.503737 6160 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503785 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503790 6160 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503793 6160 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503799 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503802 6160 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503805 6160 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503808 6160 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503810 6160 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503813 6160 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503816 6160 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503819 6160 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503822 6160 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503825 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503827 6160 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503830 6160 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503833 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503835 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503838 6160 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503841 6160 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:32:09.518600 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503843 6160 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503846 6160 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503849 6160 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503851 6160 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503854 6160 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503857 6160 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503859 6160 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503862 6160 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503864 6160 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503867 6160 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503869 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503873 6160 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503875 6160 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503878 6160 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503880 6160 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503883 6160 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503887 6160 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503890 6160 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503892 6160 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503895 6160 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:32:09.519264 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503898 6160 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503901 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503903 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503906 6160 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503909 6160 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503911 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503914 6160 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503918 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503921 6160 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503924 6160 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503926 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503929 6160 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503932 6160 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503934 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503937 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503940 6160 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503942 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503945 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503961 6160 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:32:09.519864 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503965 6160 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503968 6160 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503971 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503973 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503976 6160 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503979 6160 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503982 6160 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503985 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503987 6160 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503992 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503995 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.503997 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504000 6160 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504003 6160 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504005 6160 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504008 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504010 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504013 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504016 6160 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:32:09.520623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504018 6160 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504022 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504025 6160 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504027 6160 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504030 6160 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504033 6160 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504036 6160 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504038 6160 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504042 6160 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504044 6160 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504047 6160 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504049 6160 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504052 6160 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504055 6160 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504057 6160 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504060 6160 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504063 6160 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504065 6160 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504068 6160 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504072 6160 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:32:09.521311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504076 6160 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504079 6160 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504083 6160 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504086 6160 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504089 6160 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504091 6160 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504097 6160 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504101 6160 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:09.504104 6160 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.504183 6160 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505171 6160 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505222 6160 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505408 6160 server.go:1014] "Starting client certificate rotation" Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505576 6160 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505630 6160 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:25:58 +0000 UTC" deadline="2028-05-20 06:49:40.569756305 +0000 UTC" Oct 30 18:32:09.521972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505679 6160 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13620h17m31.064079401s" Oct 30 18:32:09.522561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.505877 6160 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:32:09.522561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.506977 6160 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:32:09.522561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.508584 6160 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:32:09.522561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.510574 6160 log.go:25] "Validated CRI v1 image API" Oct 30 18:32:09.522561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.511285 6160 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:32:09.522561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.513480 6160 fs.go:135] Filesystem UUIDs: map[2c5434fc-ef10-4e71-8d54-2813fd66abbd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b7137dc7-52cf-4708-a762-a51e82706c5c:/dev/nvme0n1p3] Oct 30 18:32:09.522758 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.513499 6160 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b/userdata/shm major:0 minor:298 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19/userdata/shm major:0 minor:397 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569/userdata/shm major:0 minor:276 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01/userdata/shm major:0 minor:274 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e/userdata/shm major:0 minor:248 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce/userdata/shm major:0 minor:356 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840/userdata/shm major:0 minor:395 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04/userdata/shm major:0 minor:324 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a/userdata/shm major:0 minor:393 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/048e9269-1abc-446c-baed-e789c6e91df0/volumes/kubernetes.io~projected/kube-api-access-gqsbf:{mountpoint:/var/lib/kubelet/pods/048e9269-1abc-446c-baed-e789c6e91df0/volumes/kubernetes.io~projected/kube-api-access-gqsbf major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/048e9269-1abc-446c-baed-e789c6e91df0/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/048e9269-1abc-446c-baed-e789c6e91df0/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~projected/kube-api-access-x6pmr:{mountpoint:/var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~projected/kube-api-access-x6pmr major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/219a3597-95e6-4836-9f38-e8091e0f0b35/volumes/kubernetes.io~projected/kube-api-access-fsfss:{mountpoint:/var/lib/kubelet/pods/219a3597-95e6-4836-9f38-e8091e0f0b35/volumes/kubernetes.io~projected/kube-api-access-fsfss major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/volumes/kubernetes.io~projected/kube-api-access-9shbt:{mountpoint:/var/lib/kubelet/pods/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/volumes/kubernetes.io~projected/kube-api-access-9shbt major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/422a1ecf-21d1-433f-8ede-d9de5a5940b7/volumes/kubernetes.io~projected/kube-api-access-gjktc:{mountpoint:/var/lib/kubelet/pods/422a1ecf-21d1-433f-8ede-d9de5a5940b7/volumes/kubernetes.io~projected/kube-api-access-gjktc major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~projected/kube-api-access-k6lh6:{mountpoint:/var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~projected/kube-api-access-k6lh6 major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/514fb142-256f-466f-aefb-dc0fbc1ce334/volumes/kubernetes.io~projected/kube-api-access-5xhxm:{mountpoint:/var/lib/kubelet/pods/514fb142-256f-466f-aefb-dc0fbc1ce334/volumes/kubernetes.io~projected/kube-api-access-5xhxm major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56d694ae-4469-45fe-b66d-36efcb67b5cb/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/56d694ae-4469-45fe-b66d-36efcb67b5cb/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:391 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6364988c-a134-41de-9d5f-38728dd0808c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/6364988c-a134-41de-9d5f-38728dd0808c/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/71968925-61e2-439b-8d7f-7d9184c7032b/volumes/kubernetes.io~projected/kube-api-access-d54g6:{mountpoint:/var/lib/kubelet/pods/71968925-61e2-439b-8d7f-7d9184c7032b/volumes/kubernetes.io~projected/kube-api-access-d54g6 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d565d10-f9d0-4450-a666-2c0805d53629/volumes/kubernetes.io~projected/kube-api-access-xcrsm:{mountpoint:/var/lib/kubelet/pods/7d565d10-f9d0-4450-a666-2c0805d53629/volumes/kubernetes.io~projected/kube-api-access-xcrsm major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d565d10-f9d0-4450-a666-2c0805d53629/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/7d565d10-f9d0-4450-a666-2c0805d53629/volumes/kubernetes.io~secret/metrics-certs major:0 minor:387 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7eb17ce4-c80f-476b-8682-c1a329bd82fb/volumes/kubernetes.io~projected/kube-api-access-bvx8s:{mountpoint:/var/lib/kubelet/pods/7eb17ce4-c80f-476b-8682-c1a329bd82fb/volumes/kubernetes.io~projected/kube-api-access-bvx8s major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~projected/kube-api-access-bvmln:{mountpoint:/var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~projected/kube-api-access-bvmln major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/alertmanager/9 major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/config-reloader/6 major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~empty-dir/config-out major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~projected/kube-api-access-r5794:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~projected/kube-api-access-r5794 major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~projected/tls-assets major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/config-volume major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/web-config major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1a0af93-464e-4b58-bf1c-ab9de497715d/volumes/kubernetes.io~projected/kube-api-access-dg97w:{mountpoint:/var/lib/kubelet/pods/a1a0af93-464e-4b58-bf1c-ab9de497715d/volumes/kubernetes.io~projected/kube-api-access-dg97w major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1a0af93-464e-4b58-bf1c-ab9de497715d/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/a1a0af93-464e-4b58-bf1c-ab9de497715d/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3635fec-23c3-49ab-8606-9803e62d959c/volumes/kubernetes.io~projected/kube-api-access-zqg5r:{mountpoint:/var/lib/kubelet/pods/a3635fec-23c3-49ab-8606-9803e62d959c/volumes/kubernetes.io~projected/kube-api-access-zqg5r major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b028c48c-de12-42cd-b596-89cef2f6308f/volumes/kubernetes.io~projected/kube-api-access-snmxz:{mountpoint:/var/lib/kubelet/pods/b028c48c-de12-42cd-b596-89cef2f6308f/volumes/kubernetes.io~projected/kube-api-access-snmxz major:0 minor:392 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~projected/kube-api-access-zxc2c:{mountpoint:/var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~projected/kube-api-access-zxc2c major:0 minor:273 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:272 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:271 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~projected/kube-api-access-qt6b7:{mountpoint:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~projected/kube-api-access-qt6b7 major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7fe0720-634c-4285-9b18-9a061f992421/volumes/kubernetes.io~projected/kube-api-access-hpfzx:{mountpoint:/var/lib/kubelet/pods/d7fe0720-634c-4285-9b18-9a061f992421/volumes/kubernetes.io~projected/kube-api-access-hpfzx major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7fe0720-634c-4285-9b18-9a061f992421/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/d7fe0720-634c-4285-9b18-9a061f992421/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volumes/kubernetes.io~projected/kube-api-access-t5h4s:{mountpoint:/var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volumes/kubernetes.io~projected/kube-api-access-t5h4s major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/90b7b240238c8c00c88805a7d7850f01ab370bf0c0f2f7125c2d82299cad2aad/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/e782e0d19abf178b275c4a7e54b0041688cf7ea26596b831223c66059f2501cf/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/3be11d7baf6b02b56e0ddde35c53c18bb5063b989b03682dab4e2f85ff93fc0a/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/7757ae80d36e5acbe53888d5c57ef8c023aade74c6f29070938d094b6339c6fe/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/7dada2e5bc61469dfd1609164b28402bb9467646baea9d2631ea63fa3cc54e98/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/7c48373858a9fb9bf93777ad4f4681a9698c581b8f7db35ec47621f4cda602b4/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/d0ab007e38b2dd08f20a8a1ed273ee4e640c3418241fe64253ab1fa73f1779db/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/06a3fa0c437df381ea2de7544a2b835e5c482306243717eb30b4ad00d28f56d7/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/012bb00b278ff764548c03aef7351b778ba2d709e636ab94e325fe69e7b3b04e/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/ea5b27f45d268bfd93afafb5ad59b9a30bbc588b1ffab6e054cf9ab3482ef324/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/7f86e7bf590379b2288a1cdadd67ded7a93d663673be068d7c8fc306309a7cb6/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/08a0bb2e21d9eaeab62b71d1c8bf45b90bed7cdda7a89a7209bc4e884f63ff02/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/e3b7b9008a18c6891ac43083e71d895ca155e3669571b184a9e716dfdbada106/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/0e6ba5acd30cced2009eb8f625cb9661cec538a6fac773ab3f4325c34383daf1/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/939a4e79c2a699a5b843dfd4e1b6c8b23b38f14e35bf5859e1048341e5d7e2ca/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/f4ef89df5658a69b2b42edb2a64e56378d14f01fa03511c37fde6b6f6d1c0cbb/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/439259119db62b72663a769a1e9ed8668e197e0c3d5c435df58f1eed654b00bc/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d4e0781b428e8eeb8aeb782cf1d6994864e2f582b7a1d71907f58f2908a722a3/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/2a65e86f0e430163f5ac94458ae72ff0149a2bcb61601d969da487453779a1ed/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/60d0f6ad5d48c51c84b0c2b490c4dd7a05394d2dc9e87b6a7c0aa887eea5bf15/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/3204d96db48dfe539d4d6ac1434ed8af8cc310ac39e76f039554441a9df717b7/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/1d56bd3698e02854a6e83304f6a149377d21b17960ddc27f8c54cc673090009b/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/22f74d7335b79c2eef52c860021732392eb2ebc8757d9d116e51a535bb67e5e2/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/64c3144915041eaecedbeb6ca701cc4b67e8178348e74942d52b6dfd262c9978/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/e06c2bb5d508f4aa26eba13e95e1f7bb50a41b90fd7c078c783e1709fd977df4/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/e3fa4f03a173f88c9ab69584e552a0088d328b875b7735eda14c0c570dde44e4/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/42f2fe1539cf8443e1f87f42883f6510a28e1ad773585274e6c3b0567d0cca48/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/60c9366a1f97607adaabd668cd276e6817baba501d08d5159726184651d6177d/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/a19627059299c430e142095614efe5d9c0dd07936497aac32b1c0af0e6071db0/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/9b0f6e635873c09d4ee79c3d36f115d79f295c9c8f2e139303e3439ba17f507c/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/197ac3ed5c9cf4fc42f1ec0010e03f5b9790ae8fde5de950bec49a31a15caec8/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/3825703092a707a870d7c9d16e4598f4b32d7541e79b0e45d3c9248541f20ab5/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/a3b10df484bd788607e94492dace5aa75a35dd3f4e99485830914f683eb26be4/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/176a70ef5ff9500962069d559dd9f1ae1a031d45d746fce0666dbcde00ba06cb/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/14d6782960b90cf4b73f18636de5be4eaedd98163c4b36a4bf3737b7329df074/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/5a550e3dc124c476da7a179c9d8bc1c982d8409c549bb1edf77d81024ea5b00a/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/51efc9b4e0ebbd1ad3b335526614301662919d30b72f30994b19447c1d153039/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/4913f8ebe479123950cbec9d1b8933069517272c3024454a7d6cee71b4d21c10/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/7b42a2d37fd09f86cd41c172b4288cf8fa9285bc2bf18e05a02d19037499324e/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/f4f924759a0897de9ebfce732eed7bafe2cf929ad2bd0ef85e2d97d8b7e27fa9/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/83304b00af54f7f3f79761ae1365503a62deecf485857fb8ed38df22e45cdc96/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/ad45e2cc05a1fd42c42b2063f19a020d67f20bac6c56e9766c9815ae0ef497e4/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/2d5ebaaf29b84c47d5e393a51bc6e00c8f938fdb997dc6cf79197da27262c410/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/227acdefaf8b0ecaab88c176220dea3c6d99efa4e32fe1449e1f1d1fbb2cb4a6/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/dd5e28823aadf6ad79ed45a91cd943244459980b074440924c9517e63a6804ef/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/9732b6eb2829a86a7882137b69c2eddffdd512b0210ff7c8773946c37baf4da3/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/8fb2edb16d03d1cd1b96fc9a8f51167039a4018b79a51468be932b5dd33a72ae/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/fcafa5c4d32c88f925dd9d6805ad7fa97da2aa5be24ad5583e3651fd75045a71/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/433c6815c397ed813cad42484087343a365e7309fdc40e8656b8af73871246a9/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/5e7f7c1500f7657e9defb526814870fe06fca4d07735e55d2cba32100c290c6a/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/dadd5ff75766c6e41fbbd4f82fc98b19a6a5d845678c4bccc4b0f9f7b7746c81/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/35846712e0fda409935b76e845b1d56e74208a94dd3e522184f0fa45c96a1d91/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-382:{mountpoint:/var/lib/containers/storage/overlay/f0959b6c7e3a67a68927ab541a62e3a1f0a8d4f62735fd92ea38e9c357bc590c/merged major:0 minor:382 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/220daddd7634d065e3af8fd2eb902217813d0a0d01e657be7fb03228cea691a2/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/ecb763a1344909089fa6f46e012a042ae6060379eb4210a1be87cd13565aebd5/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/6daee75a8f70c610ac74e2745ee6c0923fde0577a07da87365af6ca27cb61dc3/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/28c7814e316fdc48260a30b3897655bacc8e8d0923bdc1deec2d5476605a297c/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/8255832c0b8a939872a202b57486c0b0f9f48a359cc2c2128cd1a4327f13ce20/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/cc1e619beaa9fc5602e194ad94ff56baa7d4561c10c99f42f9ef92f374152a46/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/cdde3ad18282e2329ead3a1eb306bbf3478df5721449f183fa1ccfd9de48536c/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/637ccb17aa0f86feba6751a6c2335224ca6d68dafc8bed999e289a536ff3ea19/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/ed9b94ee3f9a67bd88cba971a7a026ad2fe2d760c627aa5eee2ac6e1da217230/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/81237f6a39ec8f6e1f9a252a23602b4e4170b1ad12440981eceef2a0f5351ccb/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/fe283f26d196bc19fb44d22e956eb79501112c7d110ab8825282f006ab1557b0/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/f654d7eed66f3e7f86cf0a369e9a77300086be1033aa405b6ccb8a4a8f210f2e/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/d66ae99789ad2de7886bdcf19807220e1f6ffaaf4c82835728d208970cce85b5/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/601263981a3ebfa172e2553a56bb5700e31fd49fad6a5b826786d71bfccbea7d/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/994053c58365cf55243f107fc39f4e1d16f629e72aba7064e5d31f1bc22fc142/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/179e627c5f5033ac44e97ee1912e2f6a09a6b5472bde43cb34704e5d4c5f67e6/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/1ca2d6f5443dccc6d132caf49e14deb89a6b957dc38020e47e25b4811ec31284/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 18:32:09.523016 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.522362 6160 manager.go:217] Machine: {Timestamp:2026-10-30 18:32:09.521887434 +0000 UTC m=+0.060079630 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099612 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f41c4c54a2a7516e3d32610d7b3af SystemUUID:ec2f41c4-c54a-2a75-16e3-d32610d7b3af BootID:b6958c44-a391-4f68-a13b-c624e3d89f67 Filesystems:[{Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:294 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/d7fe0720-634c-4285-9b18-9a061f992421/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/volumes/kubernetes.io~projected/kube-api-access-9shbt DeviceMajor:0 DeviceMinor:323 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:341 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6364988c-a134-41de-9d5f-38728dd0808c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:269 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7d565d10-f9d0-4450-a666-2c0805d53629/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:387 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~projected/kube-api-access-r5794 DeviceMajor:0 DeviceMinor:296 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b028c48c-de12-42cd-b596-89cef2f6308f/volumes/kubernetes.io~projected/kube-api-access-snmxz DeviceMajor:0 DeviceMinor:392 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/7eb17ce4-c80f-476b-8682-c1a329bd82fb/volumes/kubernetes.io~projected/kube-api-access-bvx8s DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:236 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/048e9269-1abc-446c-baed-e789c6e91df0/volumes/kubernetes.io~projected/kube-api-access-gqsbf DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74a651c67114af1aee8a052fc015c5ecfba688c6e928f1ec66c1c24b64bf4331/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:319 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volumes/kubernetes.io~projected/kube-api-access-t5h4s DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:292 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7d565d10-f9d0-4450-a666-2c0805d53629/volumes/kubernetes.io~projected/kube-api-access-xcrsm DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aad69d0f1ec18b076e43c9c905d69268d728e8517bc7bb23582d79d9849e3f16/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/610b03b021e744b615edc10e25e766721ced65f681cc75fa80b6af7ce020369f/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:295 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eafb104736a89d2bc1629926ad5825cc809db5ee9cb3957fe274b0cda1e9714a/userdata/shm DeviceMajor:0 DeviceMinor:393 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dfa5ea4c745099cff13c96d7329193e3a83a83ec0bf20b282f30779638e4a840/userdata/shm DeviceMajor:0 DeviceMinor:395 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a3635fec-23c3-49ab-8606-9803e62d959c/volumes/kubernetes.io~projected/kube-api-access-zqg5r DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee7661dc988f0836db2d508cc0ff8f1624041c14551f3f6981f0deeb0582409d/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:240 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/514fb142-256f-466f-aefb-dc0fbc1ce334/volumes/kubernetes.io~projected/kube-api-access-5xhxm DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c2f9d7535766068ba5db4066028720faa45576dd68486a799709258d0d082a0a/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:291 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~projected/kube-api-access-qt6b7 DeviceMajor:0 DeviceMinor:350 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/56d694ae-4469-45fe-b66d-36efcb67b5cb/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:391 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:294 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/990c1558d842fa0489594a6585160b3883ce51076b507912ffe2e8297fa471be/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a8c0b9e35bd197e67b3c7920ae6c0340aab6960b349ba415e64cee29d9a6565e/userdata/shm DeviceMajor:0 DeviceMinor:248 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:271 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:284 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a301f8507982f610c169c9a1ba7ee8d2b2a05f84ddf9d5b2e829fe6a2467372d/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/86abfef4fa662013913b9283859746fa46d85b8ff0abe463d538db78ae2af58f/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:265 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7ff36015fdaf75cf40e2205b45d35a5f660750f0cb66d1a1dee3e22308c6aabb/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:289 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/422a1ecf-21d1-433f-8ede-d9de5a5940b7/volumes/kubernetes.io~projected/kube-api-access-gjktc DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:294 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-382 DeviceMajor:0 DeviceMinor:382 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/219a3597-95e6-4836-9f38-e8091e0f0b35/volumes/kubernetes.io~projected/kube-api-access-fsfss DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:290 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:294 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93ad732af321ae301c7c315174e7f7730928355beb44a1f7145d83e187402569/userdata/shm DeviceMajor:0 DeviceMinor:276 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/efb3d9bb-411e-4587-b807-3054f555450a/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~projected/kube-api-access-bvmln DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:272 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9451ef52bf9052bb287951f155d54dff289b547b168500b9db734dd5df0f1f01/userdata/shm DeviceMajor:0 DeviceMinor:274 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b/userdata/shm DeviceMajor:0 DeviceMinor:298 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee43c5cf5c18daec1f899016844394f841d1695506e5c6318ec78f5dc7d86ea4/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:345 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/71968925-61e2-439b-8d7f-7d9184c7032b/volumes/kubernetes.io~projected/kube-api-access-d54g6 DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/048e9269-1abc-446c-baed-e789c6e91df0/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/513b3b1c-2fd2-42df-9438-4f7e52bf43bc/volumes/kubernetes.io~projected/kube-api-access-k6lh6 DeviceMajor:0 DeviceMinor:270 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d7fe0720-634c-4285-9b18-9a061f992421/volumes/kubernetes.io~projected/kube-api-access-hpfzx DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:347 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/22df52b3529f3859260e946a44d1f622c40b6fc6325f0f237141bc4143721f19/userdata/shm DeviceMajor:0 DeviceMinor:397 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b5c8f73-4e7d-4e68-939f-288190998d5a/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1eb83f0f-6234-4239-8065-df0c41d41b32/volumes/kubernetes.io~projected/kube-api-access-x6pmr DeviceMajor:0 DeviceMinor:241 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bdb2e8e3465d2a6cf0e8b1821d7f471d0e00e79d6d3291aaa18b2b1aeb9477ce/userdata/shm DeviceMajor:0 DeviceMinor:356 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bbe9260b-f403-4983-8e9a-6c328064b4c1/volumes/kubernetes.io~projected/kube-api-access-zxc2c DeviceMajor:0 DeviceMinor:273 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93a00db78a51ba83859ccf8d22cde8bfd469454d0d2ea4c4da6fbd72b9a77609/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e5200c48e97722fc918e3fb30d9c5bbcafc19358080a8ddbeb4294362edddf04/userdata/shm DeviceMajor:0 DeviceMinor:324 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a1a0af93-464e-4b58-bf1c-ab9de497715d/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c68a11ea-7b0c-414c-a1dd-a39b3b89472e/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:346 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a1a0af93-464e-4b58-bf1c-ab9de497715d/volumes/kubernetes.io~projected/kube-api-access-dg97w DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:293 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0d2e46abfafbc57 MacAddress:4a:b5:5d:88:cb:40 Speed:10000 Mtu:8901} {Name:22df52b3529f385 MacAddress:d6:a9:c1:ad:77:87 Speed:10000 Mtu:8901} {Name:610b03b021e744b MacAddress:b2:76:51:e3:69:64 Speed:10000 Mtu:8901} {Name:74a651c67114af1 MacAddress:e6:8f:8c:ad:27:10 Speed:10000 Mtu:8901} {Name:9451ef52bf9052b MacAddress:1a:3b:eb:8b:79:47 Speed:10000 Mtu:8901} {Name:a8c0b9e35bd197e MacAddress:7a:12:5f:7d:3e:ab Speed:10000 Mtu:8901} {Name:bdb2e8e3465d2a6 MacAddress:6a:23:19:a8:1b:85 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c7:c7:90:69 Speed:0 Mtu:9001} {Name:br-int MacAddress:96:44:7b:b1:41:84 Speed:0 Mtu:8901} {Name:dfa5ea4c745099c MacAddress:12:ea:07:3c:66:b8 Speed:10000 Mtu:8901} {Name:e5200c48e97722f MacAddress:e2:f5:43:0f:6e:95 Speed:10000 Mtu:8901} {Name:eafb104736a89d2 MacAddress:6e:fe:41:5b:b3:a2 Speed:10000 Mtu:8901} {Name:ee7661dc988f083 MacAddress:ae:25:f8:73:5b:1a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c7:c7:90:69 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:7a:f2:7f:1a:81:02 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:d6:eb:c6:58:89:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:32:09.523016 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.522766 6160 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:32:09.523016 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.522811 6160 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:32:09.523283 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523105 6160 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:32:09.523283 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523133 6160 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-103.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:32:09.523283 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523256 6160 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:32:09.523283 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523263 6160 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:32:09.523283 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523277 6160 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:32:09.523433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523302 6160 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:32:09.523433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523390 6160 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:32:09.523567 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523556 6160 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:32:09.523664 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523655 6160 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:32:09.523731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523689 6160 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:32:09.523731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523699 6160 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:32:09.523731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523709 6160 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:32:09.523731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523718 6160 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:32:09.523731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.523726 6160 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:32:09.524430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.524416 6160 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:32:09.524481 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.524431 6160 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:32:09.524917 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.524898 6160 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:32:09.527402 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527385 6160 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:32:09.527402 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527400 6160 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:32:09.527542 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527513 6160 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 18:32:09.527733 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527717 6160 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:32:09.527795 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527758 6160 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:32:09.527884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527873 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527892 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527902 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527911 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527921 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527931 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527941 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:32:09.527945 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527964 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:32:09.528294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527976 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:32:09.528294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527985 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:32:09.528294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.527998 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:32:09.528294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.528013 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:32:09.528294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.528047 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:32:09.528294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.528057 6160 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:32:09.531547 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.531530 6160 server.go:1263] "Started kubelet" Oct 30 18:32:09.531831 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.531609 6160 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:32:09.532122 ip-10-0-136-103 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:32:09.532259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.531618 6160 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:32:09.532259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.532194 6160 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:32:09.532747 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.532606 6160 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:32:09.533057 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.532900 6160 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:32:09.536783 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.536754 6160 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:32:09.539529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.537270 6160 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:32:09.539529 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.537701 6160 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-103.ec2.internal\" not found" Oct 30 18:32:09.539529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.537745 6160 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:32:09.539529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.537754 6160 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:32:09.539529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.538131 6160 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:32:09.541817 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.541794 6160 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:32:09.541817 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.541817 6160 factory.go:55] Registering systemd factory Oct 30 18:32:09.542032 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.541828 6160 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:32:09.542032 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.541946 6160 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:26:00 +0000 UTC" deadline="2028-07-21 22:03:09.909522494 +0000 UTC" Oct 30 18:32:09.542254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.542041 6160 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15123h31m0.367485879s" Oct 30 18:32:09.542254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.542189 6160 factory.go:153] Registering CRI-O factory Oct 30 18:32:09.542254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.542205 6160 factory.go:223] Registration of the crio container factory successfully Oct 30 18:32:09.542254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.542222 6160 factory.go:103] Registering Raw factory Oct 30 18:32:09.542254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.542237 6160 manager.go:1199] Started watching for new ooms in manager Oct 30 18:32:09.543054 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.543030 6160 manager.go:319] Starting recovery of all containers Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544453 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544490 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-federate-client-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544520 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="514fb142-256f-466f-aefb-dc0fbc1ce334" volumeName="kubernetes.io/projected/514fb142-256f-466f-aefb-dc0fbc1ce334-kube-api-access-5xhxm" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544534 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eb17ce4-c80f-476b-8682-c1a329bd82fb" volumeName="kubernetes.io/empty-dir/7eb17ce4-c80f-476b-8682-c1a329bd82fb-tmp-dir" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544547 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544571 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1a0af93-464e-4b58-bf1c-ab9de497715d" volumeName="kubernetes.io/projected/a1a0af93-464e-4b58-bf1c-ab9de497715d-kube-api-access-dg97w" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544584 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbe9260b-f403-4983-8e9a-6c328064b4c1" volumeName="kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544596 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="71968925-61e2-439b-8d7f-7d9184c7032b" volumeName="kubernetes.io/projected/71968925-61e2-439b-8d7f-7d9184c7032b-kube-api-access-d54g6" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544623 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544637 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544650 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544674 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d565d10-f9d0-4450-a666-2c0805d53629" volumeName="kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544687 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1eb83f0f-6234-4239-8065-df0c41d41b32" volumeName="kubernetes.io/configmap/1eb83f0f-6234-4239-8065-df0c41d41b32-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544713 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1a0af93-464e-4b58-bf1c-ab9de497715d" volumeName="kubernetes.io/empty-dir/a1a0af93-464e-4b58-bf1c-ab9de497715d-tmp-dir" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544742 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efb3d9bb-411e-4587-b807-3054f555450a" volumeName="kubernetes.io/projected/efb3d9bb-411e-4587-b807-3054f555450a-kube-api-access-t5h4s" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544756 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="048e9269-1abc-446c-baed-e789c6e91df0" volumeName="kubernetes.io/secret/048e9269-1abc-446c-baed-e789c6e91df0-cert" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544769 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1eb83f0f-6234-4239-8065-df0c41d41b32" volumeName="kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544791 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1eb83f0f-6234-4239-8065-df0c41d41b32" volumeName="kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544804 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="514fb142-256f-466f-aefb-dc0fbc1ce334" volumeName="kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-daemon-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544816 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544841 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="513b3b1c-2fd2-42df-9438-4f7e52bf43bc" volumeName="kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544854 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6364988c-a134-41de-9d5f-38728dd0808c" volumeName="kubernetes.io/configmap/6364988c-a134-41de-9d5f-38728dd0808c-konnectivity-ca" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544866 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d565d10-f9d0-4450-a666-2c0805d53629" volumeName="kubernetes.io/projected/7d565d10-f9d0-4450-a666-2c0805d53629-kube-api-access-xcrsm" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544889 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b5c8f73-4e7d-4e68-939f-288190998d5a" volumeName="kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-tmp" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544901 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b5c8f73-4e7d-4e68-939f-288190998d5a" volumeName="kubernetes.io/projected/8b5c8f73-4e7d-4e68-939f-288190998d5a-kube-api-access-bvmln" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544926 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbe9260b-f403-4983-8e9a-6c328064b4c1" volumeName="kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544963 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7fe0720-634c-4285-9b18-9a061f992421" volumeName="kubernetes.io/projected/d7fe0720-634c-4285-9b18-9a061f992421-kube-api-access-hpfzx" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.544978 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545009 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1eb83f0f-6234-4239-8065-df0c41d41b32" volumeName="kubernetes.io/projected/1eb83f0f-6234-4239-8065-df0c41d41b32-kube-api-access-x6pmr" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545039 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="513b3b1c-2fd2-42df-9438-4f7e52bf43bc" volumeName="kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545051 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545064 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545089 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3635fec-23c3-49ab-8606-9803e62d959c" volumeName="kubernetes.io/projected/a3635fec-23c3-49ab-8606-9803e62d959c-kube-api-access-zqg5r" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545104 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="422a1ecf-21d1-433f-8ede-d9de5a5940b7" volumeName="kubernetes.io/configmap/422a1ecf-21d1-433f-8ede-d9de5a5940b7-serviceca" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545128 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1a0af93-464e-4b58-bf1c-ab9de497715d" volumeName="kubernetes.io/configmap/a1a0af93-464e-4b58-bf1c-ab9de497715d-config-volume" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545152 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1a0af93-464e-4b58-bf1c-ab9de497715d" volumeName="kubernetes.io/secret/a1a0af93-464e-4b58-bf1c-ab9de497715d-metrics-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545164 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbe9260b-f403-4983-8e9a-6c328064b4c1" volumeName="kubernetes.io/empty-dir/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-textfile" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545176 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbe9260b-f403-4983-8e9a-6c328064b4c1" volumeName="kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545201 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545214 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7fe0720-634c-4285-9b18-9a061f992421" volumeName="kubernetes.io/empty-dir/d7fe0720-634c-4285-9b18-9a061f992421-data-volume" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545227 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="048e9269-1abc-446c-baed-e789c6e91df0" volumeName="kubernetes.io/projected/048e9269-1abc-446c-baed-e789c6e91df0-kube-api-access-gqsbf" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545266 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545280 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efb3d9bb-411e-4587-b807-3054f555450a" volumeName="kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-env-overrides" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545292 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efb3d9bb-411e-4587-b807-3054f555450a" volumeName="kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-script-lib" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545305 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efb3d9bb-411e-4587-b807-3054f555450a" volumeName="kubernetes.io/secret/efb3d9bb-411e-4587-b807-3054f555450a-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545318 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efb3d9bb-411e-4587-b807-3054f555450a" volumeName="kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545331 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="513b3b1c-2fd2-42df-9438-4f7e52bf43bc" volumeName="kubernetes.io/projected/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-api-access-k6lh6" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545343 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545356 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3635fec-23c3-49ab-8606-9803e62d959c" volumeName="kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545369 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3635fec-23c3-49ab-8606-9803e62d959c" volumeName="kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-binary-copy" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545382 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbe9260b-f403-4983-8e9a-6c328064b4c1" volumeName="kubernetes.io/projected/bbe9260b-f403-4983-8e9a-6c328064b4c1-kube-api-access-zxc2c" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545395 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7fe0720-634c-4285-9b18-9a061f992421" volumeName="kubernetes.io/configmap/d7fe0720-634c-4285-9b18-9a061f992421-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545409 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="219a3597-95e6-4836-9f38-e8091e0f0b35" volumeName="kubernetes.io/projected/219a3597-95e6-4836-9f38-e8091e0f0b35-kube-api-access-fsfss" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545421 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="514fb142-256f-466f-aefb-dc0fbc1ce334" volumeName="kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-cni-binary-copy" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545434 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b028c48c-de12-42cd-b596-89cef2f6308f" volumeName="kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545446 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-serving-certs-ca-bundle" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545485 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c8123ce-1ece-4d21-8bcb-4c7effd34d1d" volumeName="kubernetes.io/projected/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-kube-api-access-9shbt" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545510 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56d694ae-4469-45fe-b66d-36efcb67b5cb" volumeName="kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545521 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7fe0720-634c-4285-9b18-9a061f992421" volumeName="kubernetes.io/secret/d7fe0720-634c-4285-9b18-9a061f992421-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545535 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="219a3597-95e6-4836-9f38-e8091e0f0b35" volumeName="kubernetes.io/configmap/219a3597-95e6-4836-9f38-e8091e0f0b35-iptables-alerter-script" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545561 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6364988c-a134-41de-9d5f-38728dd0808c" volumeName="kubernetes.io/secret/6364988c-a134-41de-9d5f-38728dd0808c-agent-certs" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545574 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eb17ce4-c80f-476b-8682-c1a329bd82fb" volumeName="kubernetes.io/projected/7eb17ce4-c80f-476b-8682-c1a329bd82fb-kube-api-access-bvx8s" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545588 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545600 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794" seLinuxMountContext="" Oct 30 18:32:09.545578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545613 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbe9260b-f403-4983-8e9a-6c328064b4c1" volumeName="kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-accelerators-collector-config" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545626 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-client-tls" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545645 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3635fec-23c3-49ab-8606-9803e62d959c" volumeName="kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545658 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c8123ce-1ece-4d21-8bcb-4c7effd34d1d" volumeName="kubernetes.io/secret/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-monitoring-plugin-cert" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545671 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="422a1ecf-21d1-433f-8ede-d9de5a5940b7" volumeName="kubernetes.io/projected/422a1ecf-21d1-433f-8ede-d9de5a5940b7-kube-api-access-gjktc" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545687 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="513b3b1c-2fd2-42df-9438-4f7e52bf43bc" volumeName="kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545700 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="513b3b1c-2fd2-42df-9438-4f7e52bf43bc" volumeName="kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545713 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545725 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545737 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c68a11ea-7b0c-414c-a1dd-a39b3b89472e" volumeName="kubernetes.io/projected/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-kube-api-access-qt6b7" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545750 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="513b3b1c-2fd2-42df-9438-4f7e52bf43bc" volumeName="kubernetes.io/empty-dir/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-volume-directive-shadow" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545762 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b5c8f73-4e7d-4e68-939f-288190998d5a" volumeName="kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-tuned" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545786 6160 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" volumeName="kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca" seLinuxMountContext="" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545799 6160 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:32:09.548275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.545806 6160 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:32:09.555466 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.555353 6160 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:32:09.570563 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.570533 6160 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:32:09.570563 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.570562 6160 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:32:09.570739 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.570583 6160 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:32:09.570739 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.570622 6160 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:32:09.584110 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.584073 6160 generic.go:356] "Generic (PLEG): container finished" podID="bbe9260b-f403-4983-8e9a-6c328064b4c1" containerID="66dac793a90c45fa3fe5e3a86946e4a83feff2765cf0174ee18e2978cf2de266" exitCode=0 Oct 30 18:32:09.588281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.588258 6160 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 18:32:09.589830 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.589803 6160 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:09.589986 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.589965 6160 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:09.591540 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.591516 6160 generic.go:356] "Generic (PLEG): container finished" podID="2d4846e669c7fd95841df1580885fb26" containerID="6f9cad31820d824adf2709d2029330d902a5088e279780787e3572a8a4c9ac6e" exitCode=0 Oct 30 18:32:09.596705 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.596679 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf" exitCode=0 Oct 30 18:32:09.611286 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.611261 6160 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:09.612302 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.612274 6160 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:32:09.628250 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.628222 6160 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="a191e11436761d5bf68838165a6adf3e6c231ae2f02faa746177bbd648051f07" exitCode=0 Oct 30 18:32:09.628250 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.628249 6160 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="1fe3a44fa44fa851895e96d9f2f2d3b1282096d6e466b04ad8ee996593b36913" exitCode=0 Oct 30 18:32:09.628447 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.628259 6160 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="fd8a4741ef10e577a109c691d4a1bcf2f6156d4c0e84ecadefbe8cbbf67886ea" exitCode=0 Oct 30 18:32:09.628447 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.628269 6160 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="82f7b9e0735388290494b19c5f3c6a5f1409911322d5ef66cc4343c8ec565f20" exitCode=0 Oct 30 18:32:09.628447 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.628275 6160 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="1d17d86eb627d8cf937b93636214de85a0ac01fb909e0567f87ba0ee3e06d432" exitCode=0 Oct 30 18:32:09.628447 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.628281 6160 generic.go:356] "Generic (PLEG): container finished" podID="a3635fec-23c3-49ab-8606-9803e62d959c" containerID="04c8adbc9ea1c42554e8be34436367ed412f0cfe22faf37da556ddf4c91de3c7" exitCode=0 Oct 30 18:32:09.647163 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.647144 6160 manager.go:324] Recovery completed Oct 30 18:32:09.670288 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670207 6160 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:32:09.670288 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670225 6160 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:32:09.670288 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670247 6160 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:32:09.670518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670399 6160 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 18:32:09.670518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670414 6160 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 18:32:09.670518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670438 6160 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 18:32:09.670518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670495 6160 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 18:32:09.670518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670511 6160 policy_none.go:50] "Start" Oct 30 18:32:09.670740 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670526 6160 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:32:09.670740 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670541 6160 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:32:09.670740 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670731 6160 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 18:32:09.670877 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670741 6160 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 18:32:09.670877 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.670750 6160 policy_none.go:44] "Start" Oct 30 18:32:09.671001 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.670909 6160 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 18:32:09.675334 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.675307 6160 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:32:09.679215 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.679202 6160 manager.go:341] "Starting Device Plugin manager" Oct 30 18:32:09.679900 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.679228 6160 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:32:09.679988 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.679904 6160 server.go:87] "Starting device plugin registration server" Oct 30 18:32:09.680257 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680244 6160 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:32:09.680286 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680259 6160 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:32:09.680364 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680350 6160 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:32:09.680476 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680446 6160 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:32:09.680542 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680485 6160 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:32:09.680542 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680492 6160 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:32:09.680632 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.680544 6160 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:32:09.680474545Z","UUID":"e294b352-e49b-4244-8b7e-3eaf47effed2","Handler":null,"Name":"","Endpoint":""} Oct 30 18:32:09.681972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.681930 6160 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:32:09.681972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.681975 6160 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:32:09.682463 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.682441 6160 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:32:09.780427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.780385 6160 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:32:09.782435 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.782412 6160 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:32:09.782548 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.782450 6160 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:32:09.782548 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.782464 6160 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-103.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:32:09.782548 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.782517 6160 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.839616 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.839586 6160 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.839801 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.839663 6160 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.871618 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.871473 6160 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal"] Oct 30 18:32:09.872142 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872104 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" event={"ID":"2d4846e669c7fd95841df1580885fb26","Type":"ContainerStarted","Data":"e7735e8310a5348ccccf627708bc7d1876f024309ebd1b1463f776509ae2a953"} Oct 30 18:32:09.872191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872152 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" event={"ID":"2d4846e669c7fd95841df1580885fb26","Type":"ContainerDied","Data":"6f9cad31820d824adf2709d2029330d902a5088e279780787e3572a8a4c9ac6e"} Oct 30 18:32:09.872191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872172 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" event={"ID":"2d4846e669c7fd95841df1580885fb26","Type":"ContainerStarted","Data":"424fc94e35437652bd806775947c568af7114d1d84ec4a15bd4a2bc3e7d42112"} Oct 30 18:32:09.872285 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872230 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" event={"ID":"1e1514c45542b472144184b401445b5d","Type":"ContainerStarted","Data":"e1965571488baf76fc984ff2f125c81e8747cda4f655b895757e8f0c6a984e4d"} Oct 30 18:32:09.872285 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872236 6160 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.872285 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872250 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" event={"ID":"1e1514c45542b472144184b401445b5d","Type":"ContainerStarted","Data":"926a657d149feb6eaacaaaf5828f4819b121eddef68c5042ea0e3867676e3024"} Oct 30 18:32:09.872285 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.872260 6160 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.894770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.894744 6160 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:09.894909 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.894788 6160 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.895747 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.895730 6160 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:09.895787 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:09.895771 6160 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.990595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.990513 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e1514c45542b472144184b401445b5d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" (UID: \"1e1514c45542b472144184b401445b5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.990595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.990552 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:09.990595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:09.990573 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.091119 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.091081 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e1514c45542b472144184b401445b5d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" (UID: \"1e1514c45542b472144184b401445b5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.091119 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.091097 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e1514c45542b472144184b401445b5d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" (UID: \"1e1514c45542b472144184b401445b5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.091304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.091132 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.091304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.091166 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.091304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.091207 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.091304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.091235 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d4846e669c7fd95841df1580885fb26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" (UID: \"2d4846e669c7fd95841df1580885fb26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.523890 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.523832 6160 apiserver.go:52] "Watching apiserver" Oct 30 18:32:10.575142 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.575103 6160 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:32:10.575975 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.575925 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-v2hkd","openshift-multus/network-metrics-daemon-29d22","openshift-dns/node-resolver-kbnmf","openshift-monitoring/node-exporter-s76gv","openshift-ovn-kubernetes/ovnkube-node-llt5l","openshift-monitoring/prometheus-operator-9cd89b74c-rk24f","openshift-network-operator/iptables-alerter-b9875","openshift-monitoring/monitoring-plugin-77f4f76594-cft4b","openshift-monitoring/alertmanager-main-0","openshift-dns/dns-default-sxvp2","openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x","openshift-network-diagnostics/network-check-target-kvgwg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal","openshift-multus/multus-additional-cni-plugins-jxr57","openshift-multus/multus-j4m82","openshift-ingress-canary/ingress-canary-mbxqk","openshift-monitoring/telemeter-client-55f45f79df-6q4pk","kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh","openshift-insights/insights-runtime-extractor-tbzd6","openshift-image-registry/node-ca-z8j6q","kube-system/global-pull-secret-syncer-9vhcp","kube-system/konnectivity-agent-t9b8c"] Oct 30 18:32:10.599640 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.599608 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ft7c\"" Oct 30 18:32:10.600795 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.600774 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtzwt\"" Oct 30 18:32:10.600901 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.600881 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-56ppb\"" Oct 30 18:32:10.600980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.600890 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:32:10.601172 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.601158 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.601208 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.601171 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:32:10.601254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.601239 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.606100 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606080 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:32:10.606188 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606126 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:32:10.606188 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606177 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.606300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606189 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.606300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606206 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:32:10.606300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606223 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.606300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606212 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.606300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606267 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:32:10.606300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606269 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:32:10.606552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606537 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bz2z9\"" Oct 30 18:32:10.606868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606854 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ghgqv\"" Oct 30 18:32:10.606868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606858 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.607034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606987 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:32:10.607034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.606992 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xfxj4\"" Oct 30 18:32:10.607132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607105 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:32:10.607132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607112 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:32:10.607199 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607165 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:32:10.607231 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607219 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-f9b88\"" Oct 30 18:32:10.607273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607221 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:32:10.607273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607247 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:32:10.607383 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607350 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kkmm\"" Oct 30 18:32:10.607417 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607409 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:32:10.607769 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607757 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8srjr\"" Oct 30 18:32:10.607804 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607770 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtqtd\"" Oct 30 18:32:10.607920 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607907 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:32:10.607978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.607943 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsmgf\"" Oct 30 18:32:10.608060 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608047 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:32:10.608098 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608067 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:32:10.608348 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608333 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k7jl9\"" Oct 30 18:32:10.608406 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608390 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.608444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608413 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.608444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608428 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-shvwg\"" Oct 30 18:32:10.608444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608438 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.608576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608528 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.608576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608544 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:32:10.608576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.608563 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:32:10.611859 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.611843 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:32:10.612293 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.612280 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 18:32:10.612630 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.612607 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 18:32:10.612883 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.612870 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 18:32:10.612986 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.612972 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 30 18:32:10.613020 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.612980 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:32:10.613020 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.613016 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 18:32:10.613267 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.613251 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 30 18:32:10.618324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.618306 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 18:32:10.618407 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.618360 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 18:32:10.618566 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.618547 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.619197 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619181 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 18:32:10.619371 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619357 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.619427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619386 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.619615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619597 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.619615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619607 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619600 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rx4vd\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619604 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619619 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619643 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619655 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619665 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.619732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619729 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619742 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619768 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619776 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619784 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619855 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619981 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.619987 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:32:10.620030 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620025 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620090 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620102 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-d2md9\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620117 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620137 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tnxh8\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620151 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620168 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620177 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620173 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twcvb\"" Oct 30 18:32:10.620312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620171 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 18:32:10.620549 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620339 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 18:32:10.620880 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.620859 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 18:32:10.621098 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.621083 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-89zdx\"" Oct 30 18:32:10.625633 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.625610 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 18:32:10.625764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.625745 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 30 18:32:10.632036 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.632015 6160 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.632132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.632120 6160 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.639808 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.639793 6160 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:32:10.655605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.655576 6160 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:10.655743 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.655613 6160 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:32:10.655743 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:10.655633 6160 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.655743 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:10.655659 6160 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-103.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-103.ec2.internal" Oct 30 18:32:10.693937 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.693909 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb17ce4-c80f-476b-8682-c1a329bd82fb-tmp-dir\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.693937 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.693940 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/219a3597-95e6-4836-9f38-e8091e0f0b35-host-slash\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.694104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.693975 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-kubelet\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.694104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.693995 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-etc-kubernetes\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.694104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694060 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eb17ce4-c80f-476b-8682-c1a329bd82fb-tmp-dir\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.694104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694065 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.694236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694104 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-sys\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.694236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694131 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.694236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694176 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/219a3597-95e6-4836-9f38-e8091e0f0b35-iptables-alerter-script\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.694236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694208 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-conf-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.694236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694233 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxc2c\" (UniqueName: \"kubernetes.io/projected/bbe9260b-f403-4983-8e9a-6c328064b4c1-kube-api-access-zxc2c\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.694424 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694261 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-slash\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.694424 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694324 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-netns\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.694424 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694344 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.694424 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694405 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.694561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694455 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-textfile\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.694561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694482 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.694561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694514 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.694652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694557 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/219a3597-95e6-4836-9f38-e8091e0f0b35-iptables-alerter-script\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.694652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694595 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-k8s-cni-cncf-io\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.694652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694615 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.694652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694641 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:32:10.694757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694651 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.694757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694657 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-config\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.694757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694750 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.694868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694802 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-config\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.694868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694828 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.694868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694852 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-textfile\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.694868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694856 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.695080 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.694947 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-lib-modules\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.695080 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695007 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1a0af93-464e-4b58-bf1c-ab9de497715d-metrics-tls\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.695080 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695039 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.695080 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695047 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-multus\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.695080 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695044 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.695080 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695079 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695163 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-systemd\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695168 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695197 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695215 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-multus-certs\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695248 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1a0af93-464e-4b58-bf1c-ab9de497715d-metrics-tls\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695252 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvmln\" (UniqueName: \"kubernetes.io/projected/8b5c8f73-4e7d-4e68-939f-288190998d5a-kube-api-access-bvmln\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.695362 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695297 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-systemd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.695611 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695413 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-wtmp\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.695611 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695432 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.695611 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695452 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.695611 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695504 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-host\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.695723 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695633 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-ovn\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.695723 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695648 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.695723 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695673 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.695723 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695710 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.695840 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695736 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-root\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.695840 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695761 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-var-lib-kubelet\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.695840 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695786 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-kubelet-config\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.695946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695885 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-sys\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.695946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695916 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.695946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695918 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.696073 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695966 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.696073 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.695988 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6lh6\" (UniqueName: \"kubernetes.io/projected/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-api-access-k6lh6\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.696073 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696008 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-tuned\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.696073 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696026 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-systemd-units\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.696073 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696041 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcrsm\" (UniqueName: \"kubernetes.io/projected/7d565d10-f9d0-4450-a666-2c0805d53629-kube-api-access-xcrsm\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:10.696073 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696056 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg97w\" (UniqueName: \"kubernetes.io/projected/a1a0af93-464e-4b58-bf1c-ab9de497715d-kube-api-access-dg97w\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696111 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpfzx\" (UniqueName: \"kubernetes.io/projected/d7fe0720-634c-4285-9b18-9a061f992421-kube-api-access-hpfzx\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696116 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-tuned\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696147 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/422a1ecf-21d1-433f-8ede-d9de5a5940b7-serviceca\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696180 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-sys-fs\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696204 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhxm\" (UniqueName: \"kubernetes.io/projected/514fb142-256f-466f-aefb-dc0fbc1ce334-kube-api-access-5xhxm\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696228 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-metrics-client-ca\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696255 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-system-cni-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696283 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696321 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696356 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-bin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696384 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7fe0720-634c-4285-9b18-9a061f992421-crio-socket\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696442 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696475 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-netd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696494 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe9260b-f403-4983-8e9a-6c328064b4c1-metrics-client-ca\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696536 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-system-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696572 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-hostroot\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696608 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-federate-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696635 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqg5r\" (UniqueName: \"kubernetes.io/projected/a3635fec-23c3-49ab-8606-9803e62d959c-kube-api-access-zqg5r\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696652 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d565d10-f9d0-4450-a666-2c0805d53629-metrics-certs\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696658 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysconfig\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696683 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696716 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/422a1ecf-21d1-433f-8ede-d9de5a5940b7-serviceca\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.696770 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696729 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-telemeter-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696782 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d54g6\" (UniqueName: \"kubernetes.io/projected/71968925-61e2-439b-8d7f-7d9184c7032b-kube-api-access-d54g6\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696823 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-var-lib-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696841 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-etc-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696851 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-federate-client-tls\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696856 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696901 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-cnibin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696925 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6364988c-a134-41de-9d5f-38728dd0808c-agent-certs\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.696940 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-modprobe-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697001 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697033 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-kubernetes\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697057 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-conf\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697132 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697161 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsfss\" (UniqueName: \"kubernetes.io/projected/219a3597-95e6-4836-9f38-e8091e0f0b35-kube-api-access-fsfss\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697211 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6364988c-a134-41de-9d5f-38728dd0808c-agent-certs\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697233 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697244 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422a1ecf-21d1-433f-8ede-d9de5a5940b7-host\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697271 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6364988c-a134-41de-9d5f-38728dd0808c-konnectivity-ca\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697289 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-socket-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.697290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697303 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-log-socket\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697320 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697391 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697413 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697428 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697443 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697466 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-node-log\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697481 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6364988c-a134-41de-9d5f-38728dd0808c-konnectivity-ca\") pod \"konnectivity-agent-t9b8c\" (UID: \"6364988c-a134-41de-9d5f-38728dd0808c\") " pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697489 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-script-lib\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697534 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-run\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697551 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697603 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb83f0f-6234-4239-8065-df0c41d41b32-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697615 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697630 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697631 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-device-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697673 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-cni-binary-copy\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697689 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-secret-telemeter-client\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697711 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7fe0720-634c-4285-9b18-9a061f992421-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697720 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-ovnkube-script-lib\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697741 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6b7\" (UniqueName: \"kubernetes.io/projected/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-kube-api-access-qt6b7\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697768 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697785 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb83f0f-6234-4239-8065-df0c41d41b32-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697859 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-cni-binary-copy\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697860 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697895 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d7fe0720-634c-4285-9b18-9a061f992421-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697930 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-kubelet\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697980 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-bin\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.697995 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698007 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efb3d9bb-411e-4587-b807-3054f555450a-ovn-node-metrics-cert\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698046 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-registration-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698067 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-metrics-client-ca\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698084 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-cnibin\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698114 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-tmp\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.698131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698114 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698168 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698187 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b5c8f73-4e7d-4e68-939f-288190998d5a-tmp\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698213 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698240 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efb3d9bb-411e-4587-b807-3054f555450a-ovn-node-metrics-cert\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698292 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-metrics-client-ca\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698294 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7fe0720-634c-4285-9b18-9a061f992421-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698336 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-tls\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698354 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698362 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698390 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698413 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-env-overrides\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698443 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698470 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7fe0720-634c-4285-9b18-9a061f992421-data-volume\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698497 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698519 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-tls\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698525 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvx8s\" (UniqueName: \"kubernetes.io/projected/7eb17ce4-c80f-476b-8682-c1a329bd82fb-kube-api-access-bvx8s\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698540 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d7fe0720-634c-4285-9b18-9a061f992421-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698539 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3635fec-23c3-49ab-8606-9803e62d959c-cni-binary-copy\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698561 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a0af93-464e-4b58-bf1c-ab9de497715d-config-volume\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698605 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6pmr\" (UniqueName: \"kubernetes.io/projected/1eb83f0f-6234-4239-8065-df0c41d41b32-kube-api-access-x6pmr\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698603 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efb3d9bb-411e-4587-b807-3054f555450a-env-overrides\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698615 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7fe0720-634c-4285-9b18-9a061f992421-data-volume\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698617 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698626 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7eb17ce4-c80f-476b-8682-c1a329bd82fb-hosts-file\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698660 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698738 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/048e9269-1abc-446c-baed-e789c6e91df0-cert\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698775 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698779 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a0af93-464e-4b58-bf1c-ab9de497715d-config-volume\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698889 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-netns\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698923 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698978 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/048e9269-1abc-446c-baed-e789c6e91df0-cert\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.698983 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699044 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699073 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-dbus\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699098 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699119 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9shbt\" (UniqueName: \"kubernetes.io/projected/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-kube-api-access-9shbt\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699149 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699224 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-socket-dir-parent\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699244 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699264 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56d694ae-4469-45fe-b66d-36efcb67b5cb-original-pull-secret\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699262 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699276 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-daemon-config\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699317 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.699331 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699356 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqsbf\" (UniqueName: \"kubernetes.io/projected/048e9269-1abc-446c-baed-e789c6e91df0-kube-api-access-gqsbf\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699377 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699393 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699396 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-serving-certs-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699439 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699471 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5h4s\" (UniqueName: \"kubernetes.io/projected/efb3d9bb-411e-4587-b807-3054f555450a-kube-api-access-t5h4s\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699488 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-daemon-config\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699497 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a0af93-464e-4b58-bf1c-ab9de497715d-tmp-dir\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699542 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjktc\" (UniqueName: \"kubernetes.io/projected/422a1ecf-21d1-433f-8ede-d9de5a5940b7-kube-api-access-gjktc\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699567 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-serving-certs-ca-bundle\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699581 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a0af93-464e-4b58-bf1c-ab9de497715d-tmp-dir\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.700726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.699618 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1eb83f0f-6234-4239-8065-df0c41d41b32-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.724776 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.724743 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvmln\" (UniqueName: \"kubernetes.io/projected/8b5c8f73-4e7d-4e68-939f-288190998d5a-kube-api-access-bvmln\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.725799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.725765 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg97w\" (UniqueName: \"kubernetes.io/projected/a1a0af93-464e-4b58-bf1c-ab9de497715d-kube-api-access-dg97w\") pod \"dns-default-sxvp2\" (UID: \"a1a0af93-464e-4b58-bf1c-ab9de497715d\") " pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:10.726347 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.726329 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d54g6\" (UniqueName: \"kubernetes.io/projected/71968925-61e2-439b-8d7f-7d9184c7032b-kube-api-access-d54g6\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.730631 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.730610 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvx8s\" (UniqueName: \"kubernetes.io/projected/7eb17ce4-c80f-476b-8682-c1a329bd82fb-kube-api-access-bvx8s\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.730866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.730851 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9shbt\" (UniqueName: \"kubernetes.io/projected/2c8123ce-1ece-4d21-8bcb-4c7effd34d1d-kube-api-access-9shbt\") pod \"monitoring-plugin-77f4f76594-cft4b\" (UID: \"2c8123ce-1ece-4d21-8bcb-4c7effd34d1d\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:32:10.731034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.731017 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") pod \"alertmanager-main-0\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:10.731648 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.731624 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcrsm\" (UniqueName: \"kubernetes.io/projected/7d565d10-f9d0-4450-a666-2c0805d53629-kube-api-access-xcrsm\") pod \"network-metrics-daemon-29d22\" (UID: \"7d565d10-f9d0-4450-a666-2c0805d53629\") " pod="openshift-multus/network-metrics-daemon-29d22" Oct 30 18:32:10.732141 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.732125 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxc2c\" (UniqueName: \"kubernetes.io/projected/bbe9260b-f403-4983-8e9a-6c328064b4c1-kube-api-access-zxc2c\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.736389 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.736332 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqg5r\" (UniqueName: \"kubernetes.io/projected/a3635fec-23c3-49ab-8606-9803e62d959c-kube-api-access-zqg5r\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.736468 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.736410 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsfss\" (UniqueName: \"kubernetes.io/projected/219a3597-95e6-4836-9f38-e8091e0f0b35-kube-api-access-fsfss\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.736523 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.736463 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snmxz\" (UniqueName: \"kubernetes.io/projected/b028c48c-de12-42cd-b596-89cef2f6308f-kube-api-access-snmxz\") pod \"network-check-target-kvgwg\" (UID: \"b028c48c-de12-42cd-b596-89cef2f6308f\") " pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:10.736523 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.736488 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6lh6\" (UniqueName: \"kubernetes.io/projected/513b3b1c-2fd2-42df-9438-4f7e52bf43bc-kube-api-access-k6lh6\") pod \"kube-state-metrics-5dc8d7d89b-85r7x\" (UID: \"513b3b1c-2fd2-42df-9438-4f7e52bf43bc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-85r7x" Oct 30 18:32:10.736523 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.736499 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqsbf\" (UniqueName: \"kubernetes.io/projected/048e9269-1abc-446c-baed-e789c6e91df0-kube-api-access-gqsbf\") pod \"ingress-canary-mbxqk\" (UID: \"048e9269-1abc-446c-baed-e789c6e91df0\") " pod="openshift-ingress-canary/ingress-canary-mbxqk" Oct 30 18:32:10.737092 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.737073 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6pmr\" (UniqueName: \"kubernetes.io/projected/1eb83f0f-6234-4239-8065-df0c41d41b32-kube-api-access-x6pmr\") pod \"prometheus-operator-9cd89b74c-rk24f\" (UID: \"1eb83f0f-6234-4239-8065-df0c41d41b32\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-rk24f" Oct 30 18:32:10.738009 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.737994 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjktc\" (UniqueName: \"kubernetes.io/projected/422a1ecf-21d1-433f-8ede-d9de5a5940b7-kube-api-access-gjktc\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.738342 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.738325 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6b7\" (UniqueName: \"kubernetes.io/projected/c68a11ea-7b0c-414c-a1dd-a39b3b89472e-kube-api-access-qt6b7\") pod \"telemeter-client-55f45f79df-6q4pk\" (UID: \"c68a11ea-7b0c-414c-a1dd-a39b3b89472e\") " pod="openshift-monitoring/telemeter-client-55f45f79df-6q4pk" Oct 30 18:32:10.738651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.738632 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5h4s\" (UniqueName: \"kubernetes.io/projected/efb3d9bb-411e-4587-b807-3054f555450a-kube-api-access-t5h4s\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.740923 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.740902 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhxm\" (UniqueName: \"kubernetes.io/projected/514fb142-256f-466f-aefb-dc0fbc1ce334-kube-api-access-5xhxm\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.743782 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.743765 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpfzx\" (UniqueName: \"kubernetes.io/projected/d7fe0720-634c-4285-9b18-9a061f992421-kube-api-access-hpfzx\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.800722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800692 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.800722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800725 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-dbus\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800743 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800759 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-socket-dir-parent\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800778 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/219a3597-95e6-4836-9f38-e8091e0f0b35-host-slash\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800802 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800831 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-socket-dir-parent\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800837 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800849 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-kubelet\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800838 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-dbus\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800875 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-etc-kubernetes\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800885 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-kubelet\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800874 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/219a3597-95e6-4836-9f38-e8091e0f0b35-host-slash\") pod \"iptables-alerter-b9875\" (UID: \"219a3597-95e6-4836-9f38-e8091e0f0b35\") " pod="openshift-network-operator/iptables-alerter-b9875" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800896 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-sys\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800902 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-etc-kubernetes\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800919 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-conf-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800937 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-sys\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800939 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-slash\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.800976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800979 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-slash\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800983 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-netns\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.800994 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-conf-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801008 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-run-netns\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801012 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-k8s-cni-cncf-io\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801031 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-k8s-cni-cncf-io\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801041 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-lib-modules\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801080 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-multus\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801100 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-systemd\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801117 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-multus-certs\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801126 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-lib-modules\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801138 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-systemd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801155 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-multus-certs\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801163 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-wtmp\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801175 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-systemd\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801177 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-multus\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801193 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-host\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801217 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-ovn\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801224 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-node-exporter-wtmp\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801192 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-systemd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801241 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-root\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801263 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-var-lib-kubelet\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801265 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-host\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801277 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-kubelet-config\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801275 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-ovn\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801306 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-root\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801326 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56d694ae-4469-45fe-b66d-36efcb67b5cb-kubelet-config\") pod \"global-pull-secret-syncer-9vhcp\" (UID: \"56d694ae-4469-45fe-b66d-36efcb67b5cb\") " pod="kube-system/global-pull-secret-syncer-9vhcp" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801331 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-var-lib-kubelet\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801356 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-sys\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801372 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801391 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe9260b-f403-4983-8e9a-6c328064b4c1-sys\") pod \"node-exporter-s76gv\" (UID: \"bbe9260b-f403-4983-8e9a-6c328064b4c1\") " pod="openshift-monitoring/node-exporter-s76gv" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801395 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-systemd-units\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801416 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-systemd-units\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801421 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-sys-fs\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801443 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-system-cni-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801464 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801488 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-bin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801493 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801503 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-system-cni-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.801508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801513 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7fe0720-634c-4285-9b18-9a061f992421-crio-socket\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801533 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-var-lib-cni-bin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801539 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-netd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801547 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-sys-fs\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801565 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-system-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801542 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-multus-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801579 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d7fe0720-634c-4285-9b18-9a061f992421-crio-socket\") pod \"insights-runtime-extractor-tbzd6\" (UID: \"d7fe0720-634c-4285-9b18-9a061f992421\") " pod="openshift-insights/insights-runtime-extractor-tbzd6" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801589 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-hostroot\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801620 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-system-cni-dir\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801627 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-netd\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801624 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-hostroot\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801633 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysconfig\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801662 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysconfig\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801684 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801720 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-var-lib-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801752 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-etc-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801767 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801722 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801780 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-var-lib-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801784 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-cnibin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801812 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-etc-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801814 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-modprobe-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801826 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801841 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-kubernetes\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801845 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-cnibin\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801862 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-conf\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801884 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422a1ecf-21d1-433f-8ede-d9de5a5940b7-host\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801890 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-kubernetes\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801899 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-modprobe-d\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801908 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-socket-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801922 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422a1ecf-21d1-433f-8ede-d9de5a5940b7-host\") pod \"node-ca-z8j6q\" (UID: \"422a1ecf-21d1-433f-8ede-d9de5a5940b7\") " pod="openshift-image-registry/node-ca-z8j6q" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801928 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-log-socket\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801947 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-node-log\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801988 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-socket-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801989 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-log-socket\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.801987 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-run\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802019 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-node-log\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802022 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-etc-sysctl-conf\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802034 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-device-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802048 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b5c8f73-4e7d-4e68-939f-288190998d5a-run\") pod \"tuned-v2hkd\" (UID: \"8b5c8f73-4e7d-4e68-939f-288190998d5a\") " pod="openshift-cluster-node-tuning-operator/tuned-v2hkd" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802055 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-kubelet\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802069 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-bin\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802085 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-registration-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802099 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-device-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802107 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-cnibin\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802092 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-kubelet\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802130 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-host-cni-bin\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802174 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-cnibin\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802176 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71968925-61e2-439b-8d7f-7d9184c7032b-registration-dir\") pod \"aws-ebs-csi-driver-node-2nsvh\" (UID: \"71968925-61e2-439b-8d7f-7d9184c7032b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nsvh" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802189 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802215 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802233 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7eb17ce4-c80f-476b-8682-c1a329bd82fb-hosts-file\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802250 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-netns\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802259 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efb3d9bb-411e-4587-b807-3054f555450a-run-openvswitch\") pod \"ovnkube-node-llt5l\" (UID: \"efb3d9bb-411e-4587-b807-3054f555450a\") " pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802283 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3635fec-23c3-49ab-8606-9803e62d959c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jxr57\" (UID: \"a3635fec-23c3-49ab-8606-9803e62d959c\") " pod="openshift-multus/multus-additional-cni-plugins-jxr57" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802296 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7eb17ce4-c80f-476b-8682-c1a329bd82fb-hosts-file\") pod \"node-resolver-kbnmf\" (UID: \"7eb17ce4-c80f-476b-8682-c1a329bd82fb\") " pod="openshift-dns/node-resolver-kbnmf" Oct 30 18:32:10.802626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:10.802318 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/514fb142-256f-466f-aefb-dc0fbc1ce334-host-run-netns\") pod \"multus-j4m82\" (UID: \"514fb142-256f-466f-aefb-dc0fbc1ce334\") " pod="openshift-multus/multus-j4m82" Oct 30 18:32:11.125598 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:11.125517 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:12.371437 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:12.371405 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sxvp2" Oct 30 18:32:14.394754 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:14.394716 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:32:15.719715 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:15.719686 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:16.534210 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:16.534173 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:16.653432 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:16.653392 6160 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:32:17.026904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:17.026820 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cft4b" Oct 30 18:32:20.837798 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:20.837609 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-llt5l" Oct 30 18:32:39.615782 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.615750 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t9b8c" Oct 30 18:32:39.765633 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.765596 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:32:39.766147 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.766088 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="alertmanager" containerID="cri-o://8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb" gracePeriod=120 Oct 30 18:32:39.766340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.766146 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-web" containerID="cri-o://7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156" gracePeriod=120 Oct 30 18:32:39.766340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.766181 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-metric" containerID="cri-o://4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c" gracePeriod=120 Oct 30 18:32:39.766340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.766259 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="config-reloader" containerID="cri-o://1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7" gracePeriod=120 Oct 30 18:32:39.766674 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.766365 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="prom-label-proxy" containerID="cri-o://19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a" gracePeriod=120 Oct 30 18:32:39.766674 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:39.766466 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy" containerID="cri-o://6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b" gracePeriod=120 Oct 30 18:32:39.802272 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:39.802237 6160 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbf3cae_5e36_4c89_adf9_32f652e9b89f.slice/crio-1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7.scope\": RecentStats: unable to find data in memory cache]" Oct 30 18:32:40.736678 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736644 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a" exitCode=0 Oct 30 18:32:40.736678 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736671 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b" exitCode=0 Oct 30 18:32:40.736678 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736679 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7" exitCode=0 Oct 30 18:32:40.736678 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736685 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb" exitCode=0 Oct 30 18:32:40.737207 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736722 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a"} Oct 30 18:32:40.737207 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736790 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b"} Oct 30 18:32:40.737207 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736816 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7"} Oct 30 18:32:40.737207 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:40.736838 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb"} Oct 30 18:32:41.041907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.041880 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.119109 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119076 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119121 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119153 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119187 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119219 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119250 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119288 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119345 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119383 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119412 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119463 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119490 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.119552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.119520 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\" (UID: \"8cbf3cae-5e36-4c89-adf9-32f652e9b89f\") " Oct 30 18:32:41.122039 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.121947 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:41.122244 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.122158 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:41.122419 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.122394 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:41.122699 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.122664 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:41.122922 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.122874 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.123069 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.123038 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.123196 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.123097 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:41.123261 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.123203 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.123400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.123373 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.123498 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.123473 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "kube-api-access-r5794". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:41.123612 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.123594 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.126392 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.126255 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.140723 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.140681 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config" pod "8cbf3cae-5e36-4c89-adf9-32f652e9b89f" (UID: "8cbf3cae-5e36-4c89-adf9-32f652e9b89f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:41.220628 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220586 6160 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-main-db\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220628 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220623 6160 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-tls-assets\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220628 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220637 6160 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220680 6160 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220694 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-metrics-client-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220706 6160 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220720 6160 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-out\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220732 6160 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-cluster-tls-config\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220744 6160 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220756 6160 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-secret-alertmanager-main-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220768 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5794\" (UniqueName: \"kubernetes.io/projected/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-kube-api-access-r5794\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220779 6160 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.220907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.220791 6160 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cbf3cae-5e36-4c89-adf9-32f652e9b89f-web-config\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:32:41.638584 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.638551 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kvgwg" Oct 30 18:32:41.743768 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743733 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c" exitCode=0 Oct 30 18:32:41.743768 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743760 6160 generic.go:356] "Generic (PLEG): container finished" podID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerID="7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156" exitCode=0 Oct 30 18:32:41.744340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743792 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c"} Oct 30 18:32:41.744340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743826 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.744340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743870 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156"} Oct 30 18:32:41.744340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743919 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8cbf3cae-5e36-4c89-adf9-32f652e9b89f","Type":"ContainerDied","Data":"0d2e46abfafbc57763ed8ceb96e310eb263ae51b9ca240a55acd66ff01ef601b"} Oct 30 18:32:41.744340 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.743939 6160 scope.go:122] "RemoveContainer" containerID="19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a" Oct 30 18:32:41.764607 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.764580 6160 scope.go:122] "RemoveContainer" containerID="4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c" Oct 30 18:32:41.771728 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.771697 6160 scope.go:122] "RemoveContainer" containerID="6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b" Oct 30 18:32:41.778671 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.778652 6160 scope.go:122] "RemoveContainer" containerID="7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156" Oct 30 18:32:41.791948 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.791924 6160 scope.go:122] "RemoveContainer" containerID="1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7" Oct 30 18:32:41.798828 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.798804 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:32:41.799138 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.799120 6160 scope.go:122] "RemoveContainer" containerID="8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb" Oct 30 18:32:41.806307 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.806286 6160 scope.go:122] "RemoveContainer" containerID="5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf" Oct 30 18:32:41.810722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.810701 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:32:41.813878 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.813859 6160 scope.go:122] "RemoveContainer" containerID="19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a" Oct 30 18:32:41.814206 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.814183 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a\": container with ID starting with 19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a not found: ID does not exist" containerID="19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a" Oct 30 18:32:41.814269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.814217 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a"} err="failed to get container status \"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a\": rpc error: code = NotFound desc = could not find container \"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a\": container with ID starting with 19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a not found: ID does not exist" Oct 30 18:32:41.814269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.814254 6160 scope.go:122] "RemoveContainer" containerID="4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c" Oct 30 18:32:41.814533 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.814506 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c\": container with ID starting with 4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c not found: ID does not exist" containerID="4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c" Oct 30 18:32:41.814576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.814541 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c"} err="failed to get container status \"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c\": rpc error: code = NotFound desc = could not find container \"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c\": container with ID starting with 4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c not found: ID does not exist" Oct 30 18:32:41.814576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.814558 6160 scope.go:122] "RemoveContainer" containerID="6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b" Oct 30 18:32:41.814802 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.814783 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b\": container with ID starting with 6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b not found: ID does not exist" containerID="6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b" Oct 30 18:32:41.814851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.814808 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b"} err="failed to get container status \"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b\": rpc error: code = NotFound desc = could not find container \"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b\": container with ID starting with 6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b not found: ID does not exist" Oct 30 18:32:41.814851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.814823 6160 scope.go:122] "RemoveContainer" containerID="7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156" Oct 30 18:32:41.815067 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.815050 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156\": container with ID starting with 7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156 not found: ID does not exist" containerID="7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156" Oct 30 18:32:41.815124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815073 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156"} err="failed to get container status \"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156\": rpc error: code = NotFound desc = could not find container \"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156\": container with ID starting with 7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156 not found: ID does not exist" Oct 30 18:32:41.815124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815087 6160 scope.go:122] "RemoveContainer" containerID="1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7" Oct 30 18:32:41.815347 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.815329 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7\": container with ID starting with 1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7 not found: ID does not exist" containerID="1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7" Oct 30 18:32:41.815420 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815362 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7"} err="failed to get container status \"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7\": rpc error: code = NotFound desc = could not find container \"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7\": container with ID starting with 1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7 not found: ID does not exist" Oct 30 18:32:41.815420 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815382 6160 scope.go:122] "RemoveContainer" containerID="8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb" Oct 30 18:32:41.815611 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.815594 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb\": container with ID starting with 8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb not found: ID does not exist" containerID="8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb" Oct 30 18:32:41.815651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815619 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb"} err="failed to get container status \"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb\": rpc error: code = NotFound desc = could not find container \"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb\": container with ID starting with 8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb not found: ID does not exist" Oct 30 18:32:41.815651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815632 6160 scope.go:122] "RemoveContainer" containerID="5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf" Oct 30 18:32:41.815870 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:41.815852 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf\": container with ID starting with 5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf not found: ID does not exist" containerID="5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf" Oct 30 18:32:41.815916 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815876 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf"} err="failed to get container status \"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf\": rpc error: code = NotFound desc = could not find container \"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf\": container with ID starting with 5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf not found: ID does not exist" Oct 30 18:32:41.815916 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.815890 6160 scope.go:122] "RemoveContainer" containerID="19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a" Oct 30 18:32:41.816113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816096 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a"} err="failed to get container status \"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a\": rpc error: code = NotFound desc = could not find container \"19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a\": container with ID starting with 19722f794498f49b3b331644bdccc216034c838bc759384a18b1f5878fe2eb9a not found: ID does not exist" Oct 30 18:32:41.816154 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816114 6160 scope.go:122] "RemoveContainer" containerID="4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c" Oct 30 18:32:41.816330 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816310 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c"} err="failed to get container status \"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c\": rpc error: code = NotFound desc = could not find container \"4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c\": container with ID starting with 4c455fd5659d3462250192b8fae50bbdb481588d78b005a86b8923b6a769547c not found: ID does not exist" Oct 30 18:32:41.816400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816331 6160 scope.go:122] "RemoveContainer" containerID="6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b" Oct 30 18:32:41.816574 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816556 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b"} err="failed to get container status \"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b\": rpc error: code = NotFound desc = could not find container \"6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b\": container with ID starting with 6f11a6ed9990bd595c5968121c8f3fd871aef7cf3440e7e9691fb19ce626252b not found: ID does not exist" Oct 30 18:32:41.816623 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816574 6160 scope.go:122] "RemoveContainer" containerID="7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156" Oct 30 18:32:41.816797 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816778 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156"} err="failed to get container status \"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156\": rpc error: code = NotFound desc = could not find container \"7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156\": container with ID starting with 7cc48060667adf8c267cd33d2d0a08709b802af114b5521f2c52cdd16ab07156 not found: ID does not exist" Oct 30 18:32:41.816873 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.816798 6160 scope.go:122] "RemoveContainer" containerID="1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7" Oct 30 18:32:41.817062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.817044 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7"} err="failed to get container status \"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7\": rpc error: code = NotFound desc = could not find container \"1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7\": container with ID starting with 1e7290b280e4d1752af61b9095f19bb4a0801455203967fcd8c5c7ceb78150b7 not found: ID does not exist" Oct 30 18:32:41.817123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.817062 6160 scope.go:122] "RemoveContainer" containerID="8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb" Oct 30 18:32:41.817293 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.817275 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb"} err="failed to get container status \"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb\": rpc error: code = NotFound desc = could not find container \"8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb\": container with ID starting with 8897b27462b2d3b8cfb86d84db0f1225339c2ef38fdc48ab5944a572acc43edb not found: ID does not exist" Oct 30 18:32:41.817336 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.817293 6160 scope.go:122] "RemoveContainer" containerID="5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf" Oct 30 18:32:41.817529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.817507 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf"} err="failed to get container status \"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf\": rpc error: code = NotFound desc = could not find container \"5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf\": container with ID starting with 5ad1256ba90124c5750a90a9fcd29afcccf29ffeb16d65205c8c9d1268d909bf not found: ID does not exist" Oct 30 18:32:41.834333 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834261 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:32:41.834526 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834513 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="alertmanager" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834527 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="alertmanager" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834538 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="config-reloader" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834543 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="config-reloader" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834558 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834563 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834568 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="init-config-reloader" Oct 30 18:32:41.834571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834573 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="init-config-reloader" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834581 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="prom-label-proxy" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834586 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="prom-label-proxy" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834591 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-metric" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834596 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-metric" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834604 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-web" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834609 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-web" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834649 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="alertmanager" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834657 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="config-reloader" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834665 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834670 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="init-config-reloader" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834675 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="prom-label-proxy" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834680 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-metric" Oct 30 18:32:41.834791 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.834685 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" containerName="kube-rbac-proxy-web" Oct 30 18:32:41.849465 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.849433 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.866267 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866242 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 18:32:41.866267 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866254 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 18:32:41.866267 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866243 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 18:32:41.866540 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866359 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 18:32:41.866540 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866358 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 18:32:41.866540 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866428 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 18:32:41.866687 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866599 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tnxh8\"" Oct 30 18:32:41.866687 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.866637 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 18:32:41.878499 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.878440 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 18:32:41.878635 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.878595 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 18:32:41.878942 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.878923 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:32:41.925681 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925633 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-config-volume\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.925681 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925687 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.925904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925707 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.925904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925767 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.925904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925837 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.925904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925852 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwp2m\" (UniqueName: \"kubernetes.io/projected/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-kube-api-access-hwp2m\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.925904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925869 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-web-config\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.926085 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.925895 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-config-out\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.926085 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.926031 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.926085 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.926061 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.926179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.926082 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.926179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.926165 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:41.926284 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:41.926190 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027250 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027204 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-config-out\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027250 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027256 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027383 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027422 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027466 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027508 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027507 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027720 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027535 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-config-volume\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027720 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027564 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.027720 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.027656 6160 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:32:42.028305 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.028278 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.028415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.028344 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.028415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.028381 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.028527 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.028454 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.028527 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.028480 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwp2m\" (UniqueName: \"kubernetes.io/projected/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-kube-api-access-hwp2m\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.028527 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.028506 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-web-config\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.030421 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.030400 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-config-out\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.030545 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.030525 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.030731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.030707 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.030731 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.030722 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-config-volume\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.030868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.030803 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.031050 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.031031 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.031186 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.031166 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-web-config\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.031475 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.031449 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.031554 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.031538 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.031739 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.031718 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.032564 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.032548 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.054767 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.054736 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwp2m\" (UniqueName: \"kubernetes.io/projected/f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9-kube-api-access-hwp2m\") pod \"alertmanager-main-0\" (UID: \"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.206651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.206563 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 18:32:42.566554 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.566513 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 18:32:42.583360 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:32:42.583311 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c92ce6_75ab_4c1e_bf93_d15d70abd3f9.slice/crio-bc533020758af5d5aa4effe2bf62f5988f16b58d3efcadac3cde40b4d0aad008 WatchSource:0}: Error finding container bc533020758af5d5aa4effe2bf62f5988f16b58d3efcadac3cde40b4d0aad008: Status 404 returned error can't find the container with id bc533020758af5d5aa4effe2bf62f5988f16b58d3efcadac3cde40b4d0aad008 Oct 30 18:32:42.753125 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.753091 6160 generic.go:356] "Generic (PLEG): container finished" podID="f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9" containerID="e01e87961b94cc87052de4f76b326be24628287bd8687393c3eca4d0fbdc1e9c" exitCode=0 Oct 30 18:32:42.753556 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.753133 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerDied","Data":"e01e87961b94cc87052de4f76b326be24628287bd8687393c3eca4d0fbdc1e9c"} Oct 30 18:32:42.753556 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:42.753206 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"bc533020758af5d5aa4effe2bf62f5988f16b58d3efcadac3cde40b4d0aad008"} Oct 30 18:32:43.575859 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.575817 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cbf3cae-5e36-4c89-adf9-32f652e9b89f" path="/var/lib/kubelet/pods/8cbf3cae-5e36-4c89-adf9-32f652e9b89f/volumes" Oct 30 18:32:43.759272 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.759237 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"78424095837accfb2b727689421111a9c665fa7d64f6554dc92b057e005cfa45"} Oct 30 18:32:43.759696 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.759302 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"8786740bc169f6cc99c75302e6eabd1f85ef532ce16fa6a9909cdd12b17dda62"} Oct 30 18:32:43.759696 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.759328 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"3e4730f91b321faedb490e155f4f7b70ab2554ae74818af3098b909397875ebc"} Oct 30 18:32:43.759696 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.759363 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"8d944723bb44451e9376fcc904a28e04398c1e820a276db0e3c06e9846b30c8b"} Oct 30 18:32:43.759696 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.759386 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"029a4197255e147cea882c3818397aa9c1cd4917adb794900da993d1ae4cd653"} Oct 30 18:32:43.759696 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.759410 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9","Type":"ContainerStarted","Data":"fb11092ecc64b9d06cb2c5c3a17afe2afc591f6883c24b27f703aa2edb0e1657"} Oct 30 18:32:43.823529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:32:43.823465 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.812010566 podStartE2EDuration="2.823453132s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.442566ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:32:43.822518358 +0000 UTC m=+34.360710563" watchObservedRunningTime="2026-10-30 18:32:43.823453132 +0000 UTC m=+34.361645336" Oct 30 18:32:44.403555 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:32:44.403519 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:33:14.395155 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:33:14.395109 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:33:44.396283 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:33:44.396241 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:34:14.396091 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:34:14.396054 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:34:44.395826 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:34:44.395789 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:35:14.394788 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:35:14.394752 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:35:15.381732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.381703 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd"] Oct 30 18:35:15.384050 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.384029 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.410261 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.410241 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Oct 30 18:35:15.410261 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.410240 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Oct 30 18:35:15.410606 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.410517 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-88w5n\"" Oct 30 18:35:15.435524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.435498 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd"] Oct 30 18:35:15.464547 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.464514 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbn58\" (UniqueName: \"kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.464547 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.464553 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.464752 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.464682 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.565191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.565157 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbn58\" (UniqueName: \"kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.565376 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.565197 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.565376 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.565245 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.565579 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.565564 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.565620 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.565606 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.585555 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.585528 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk"] Oct 30 18:35:15.588081 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.588063 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.598155 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.598131 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbn58\" (UniqueName: \"kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.612671 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.612645 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk"] Oct 30 18:35:15.633795 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.633745 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-m5nwz\"" Oct 30 18:35:15.666310 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.666289 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.666409 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.666344 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbnwr\" (UniqueName: \"kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.666491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.666476 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.696324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.696299 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:15.767604 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.767572 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.767764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.767621 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.767764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.767660 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbnwr\" (UniqueName: \"kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.768405 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.768375 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.768405 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.768397 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.794104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.794074 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbnwr\" (UniqueName: \"kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.885432 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.885349 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz"] Oct 30 18:35:15.887941 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.887920 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.899819 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.899800 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:15.906217 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.906196 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:35:15.909237 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.909216 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 30 18:35:15.909317 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.909250 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 30 18:35:15.909317 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.909268 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:35:15.910067 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.910028 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 30 18:35:15.913724 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.910180 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bjqk2\"" Oct 30 18:35:15.913724 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.910212 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 18:35:15.915265 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.915243 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz"] Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970421 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970521 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970555 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970587 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970629 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970667 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b6bk\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.971146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.970717 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:15.999903 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:15.999872 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd"] Oct 30 18:35:16.004891 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:35:16.004867 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd288b9c0_e29d_4728_a69b_eee3be5a5d34.slice/crio-eabf85f989cf7e262b82752c924fae4383721b53173e94f2e134e7d86281b45c WatchSource:0}: Error finding container eabf85f989cf7e262b82752c924fae4383721b53173e94f2e134e7d86281b45c: Status 404 returned error can't find the container with id eabf85f989cf7e262b82752c924fae4383721b53173e94f2e134e7d86281b45c Oct 30 18:35:16.006764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.006747 6160 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:35:16.072024 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.071995 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072121 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072056 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072121 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072084 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072121 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072112 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072287 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072150 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072287 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072175 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b6bk\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072287 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072223 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.072849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.072824 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.074400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.074373 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.074531 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.074513 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.074606 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.074543 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.074669 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.074644 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.094643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.094623 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.094717 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.094659 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b6bk\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk\") pod \"istiod-openshift-gateway-7876bfcb74-skcpz\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.193809 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.193715 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk"] Oct 30 18:35:16.197917 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:35:16.197897 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3112f526_2d47_4df0_a390_43518517d245.slice/crio-0d156023b73b24ca1b7fafbf698414759cef4e69850e2afa0f06abfadf78aace WatchSource:0}: Error finding container 0d156023b73b24ca1b7fafbf698414759cef4e69850e2afa0f06abfadf78aace: Status 404 returned error can't find the container with id 0d156023b73b24ca1b7fafbf698414759cef4e69850e2afa0f06abfadf78aace Oct 30 18:35:16.198659 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.198644 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:16.264943 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.264903 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" event={"ID":"3112f526-2d47-4df0-a390-43518517d245","Type":"ContainerStarted","Data":"0d156023b73b24ca1b7fafbf698414759cef4e69850e2afa0f06abfadf78aace"} Oct 30 18:35:16.266478 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.266444 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" event={"ID":"d288b9c0-e29d-4728-a69b-eee3be5a5d34","Type":"ContainerStarted","Data":"eabf85f989cf7e262b82752c924fae4383721b53173e94f2e134e7d86281b45c"} Oct 30 18:35:16.503083 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:16.502992 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz"] Oct 30 18:35:16.509130 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:35:16.509096 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302d31fc_2367_40b4_9d85_a702077783ea.slice/crio-66188ba2752083b599e56b719f95f0adadd60797e6123ab1c8551061b5b90afc WatchSource:0}: Error finding container 66188ba2752083b599e56b719f95f0adadd60797e6123ab1c8551061b5b90afc: Status 404 returned error can't find the container with id 66188ba2752083b599e56b719f95f0adadd60797e6123ab1c8551061b5b90afc Oct 30 18:35:17.271981 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:17.271914 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" event={"ID":"302d31fc-2367-40b4-9d85-a702077783ea","Type":"ContainerStarted","Data":"66188ba2752083b599e56b719f95f0adadd60797e6123ab1c8551061b5b90afc"} Oct 30 18:35:19.963361 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:19.963322 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:35:19.963708 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:19.963388 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:35:20.291806 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:20.291770 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" event={"ID":"302d31fc-2367-40b4-9d85-a702077783ea","Type":"ContainerStarted","Data":"6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31"} Oct 30 18:35:20.292088 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:20.292059 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:20.366223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:20.366158 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" podStartSLOduration=1.915318597 podStartE2EDuration="5.366140011s" totalImagesPullingTime="3.450821414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:20.354572282 +0000 UTC m=+190.892764489" watchObservedRunningTime="2026-10-30 18:35:20.366140011 +0000 UTC m=+190.904332216" Oct 30 18:35:21.299071 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:21.298662 6160 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-skcpz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:35:21.299071 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:21.298722 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" podUID="302d31fc-2367-40b4-9d85-a702077783ea" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:35:23.307459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:23.307419 6160 generic.go:356] "Generic (PLEG): container finished" podID="3112f526-2d47-4df0-a390-43518517d245" containerID="07fb8bd6481d4672c63d90fe9a100d3437de3f373248a0168f6951775a3facb1" exitCode=0 Oct 30 18:35:23.307895 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:23.307509 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" event={"ID":"3112f526-2d47-4df0-a390-43518517d245","Type":"ContainerDied","Data":"07fb8bd6481d4672c63d90fe9a100d3437de3f373248a0168f6951775a3facb1"} Oct 30 18:35:23.309207 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:23.309182 6160 generic.go:356] "Generic (PLEG): container finished" podID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerID="33ba743f3a8626028fdc8c1295b10341388c9a6a3c3d5097ed9640161a3901ec" exitCode=0 Oct 30 18:35:23.309287 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:23.309222 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" event={"ID":"d288b9c0-e29d-4728-a69b-eee3be5a5d34","Type":"ContainerDied","Data":"33ba743f3a8626028fdc8c1295b10341388c9a6a3c3d5097ed9640161a3901ec"} Oct 30 18:35:24.298335 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:24.298298 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:35:24.315374 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:24.315332 6160 generic.go:356] "Generic (PLEG): container finished" podID="3112f526-2d47-4df0-a390-43518517d245" containerID="2ff0e7daca3149b230745c31bf4dc0f602579b96269fdb753760fa4901600b38" exitCode=0 Oct 30 18:35:24.315805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:24.315425 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" event={"ID":"3112f526-2d47-4df0-a390-43518517d245","Type":"ContainerDied","Data":"2ff0e7daca3149b230745c31bf4dc0f602579b96269fdb753760fa4901600b38"} Oct 30 18:35:26.326114 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:26.326074 6160 generic.go:356] "Generic (PLEG): container finished" podID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerID="7e5fbb530981bd4334729bd354f925c3cad2e76df51409724ecebff7b32a6bff" exitCode=0 Oct 30 18:35:26.326541 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:26.326164 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" event={"ID":"d288b9c0-e29d-4728-a69b-eee3be5a5d34","Type":"ContainerDied","Data":"7e5fbb530981bd4334729bd354f925c3cad2e76df51409724ecebff7b32a6bff"} Oct 30 18:35:31.351800 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:31.351758 6160 generic.go:356] "Generic (PLEG): container finished" podID="3112f526-2d47-4df0-a390-43518517d245" containerID="f14bdced7111ab60007c152b5823a93be60b639c17889dedf1fddbadc2704771" exitCode=0 Oct 30 18:35:31.352326 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:31.351825 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" event={"ID":"3112f526-2d47-4df0-a390-43518517d245","Type":"ContainerDied","Data":"f14bdced7111ab60007c152b5823a93be60b639c17889dedf1fddbadc2704771"} Oct 30 18:35:32.487800 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.487772 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:32.624977 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.624913 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr\" (UniqueName: \"kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr\") pod \"3112f526-2d47-4df0-a390-43518517d245\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " Oct 30 18:35:32.625119 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.625006 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util\") pod \"3112f526-2d47-4df0-a390-43518517d245\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " Oct 30 18:35:32.625119 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.625099 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle\") pod \"3112f526-2d47-4df0-a390-43518517d245\" (UID: \"3112f526-2d47-4df0-a390-43518517d245\") " Oct 30 18:35:32.625756 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.625730 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle" pod "3112f526-2d47-4df0-a390-43518517d245" (UID: "3112f526-2d47-4df0-a390-43518517d245"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:32.627196 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.627163 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr" pod "3112f526-2d47-4df0-a390-43518517d245" (UID: "3112f526-2d47-4df0-a390-43518517d245"). InnerVolumeSpecName "kube-api-access-jbnwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:32.629270 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.629246 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util" pod "3112f526-2d47-4df0-a390-43518517d245" (UID: "3112f526-2d47-4df0-a390-43518517d245"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:32.726159 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.726121 6160 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-util\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:35:32.726159 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.726151 6160 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3112f526-2d47-4df0-a390-43518517d245-bundle\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:35:32.726159 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:32.726164 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbnwr\" (UniqueName: \"kubernetes.io/projected/3112f526-2d47-4df0-a390-43518517d245-kube-api-access-jbnwr\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:35:33.361895 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:33.361862 6160 generic.go:356] "Generic (PLEG): container finished" podID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerID="60bd080b3863347f03ec42b0164a3b8f4d9d60e8b0d70e60dcc8d1810fbf5b22" exitCode=0 Oct 30 18:35:33.362107 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:33.361939 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" event={"ID":"d288b9c0-e29d-4728-a69b-eee3be5a5d34","Type":"ContainerDied","Data":"60bd080b3863347f03ec42b0164a3b8f4d9d60e8b0d70e60dcc8d1810fbf5b22"} Oct 30 18:35:33.363777 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:33.363748 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" event={"ID":"3112f526-2d47-4df0-a390-43518517d245","Type":"ContainerDied","Data":"0d156023b73b24ca1b7fafbf698414759cef4e69850e2afa0f06abfadf78aace"} Oct 30 18:35:33.363777 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:33.363771 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ddmmk" Oct 30 18:35:33.363920 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:33.363794 6160 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d156023b73b24ca1b7fafbf698414759cef4e69850e2afa0f06abfadf78aace" Oct 30 18:35:34.488834 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.488807 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:34.541216 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.541188 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle\") pod \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " Oct 30 18:35:34.541400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.541227 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util\") pod \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " Oct 30 18:35:34.541400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.541295 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58\" (UniqueName: \"kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58\") pod \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\" (UID: \"d288b9c0-e29d-4728-a69b-eee3be5a5d34\") " Oct 30 18:35:34.541691 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.541657 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle" pod "d288b9c0-e29d-4728-a69b-eee3be5a5d34" (UID: "d288b9c0-e29d-4728-a69b-eee3be5a5d34"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:34.543606 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.543577 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58" pod "d288b9c0-e29d-4728-a69b-eee3be5a5d34" (UID: "d288b9c0-e29d-4728-a69b-eee3be5a5d34"). InnerVolumeSpecName "kube-api-access-rbn58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:35:34.547262 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.547222 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util" pod "d288b9c0-e29d-4728-a69b-eee3be5a5d34" (UID: "d288b9c0-e29d-4728-a69b-eee3be5a5d34"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:35:34.642304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.642211 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbn58\" (UniqueName: \"kubernetes.io/projected/d288b9c0-e29d-4728-a69b-eee3be5a5d34-kube-api-access-rbn58\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:35:34.642304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.642258 6160 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-bundle\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:35:34.642304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:34.642268 6160 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d288b9c0-e29d-4728-a69b-eee3be5a5d34-util\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:35:35.372098 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:35.372072 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" Oct 30 18:35:35.372242 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:35.372064 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644f8nzd" event={"ID":"d288b9c0-e29d-4728-a69b-eee3be5a5d34","Type":"ContainerDied","Data":"eabf85f989cf7e262b82752c924fae4383721b53173e94f2e134e7d86281b45c"} Oct 30 18:35:35.372242 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:35.372192 6160 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eabf85f989cf7e262b82752c924fae4383721b53173e94f2e134e7d86281b45c" Oct 30 18:35:44.396270 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:35:44.396228 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:35:58.882123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882085 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v"] Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882389 6160 cpu_manager.go:414] "containerMap: removing container" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="pull" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882400 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="pull" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882412 6160 cpu_manager.go:414] "containerMap: removing container" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="util" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882417 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="util" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882443 6160 cpu_manager.go:414] "containerMap: removing container" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="pull" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882448 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="pull" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882454 6160 cpu_manager.go:414] "containerMap: removing container" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="extract" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882458 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="extract" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882466 6160 cpu_manager.go:414] "containerMap: removing container" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="util" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882470 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="util" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882511 6160 cpu_manager.go:414] "containerMap: removing container" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="extract" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882516 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="extract" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882589 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="3112f526-2d47-4df0-a390-43518517d245" containerName="extract" Oct 30 18:35:58.883452 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.882603 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="d288b9c0-e29d-4728-a69b-eee3be5a5d34" containerName="extract" Oct 30 18:35:58.884643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.884628 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.940029 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.939986 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v"] Oct 30 18:35:58.946361 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.946324 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg5sq\" (UniqueName: \"kubernetes.io/projected/ee9b04a4-387d-4702-b4ad-9a273642e191-kube-api-access-cg5sq\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.946541 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.946385 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.946541 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.946482 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.946541 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.946519 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.946664 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.946598 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ee9b04a4-387d-4702-b4ad-9a273642e191-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.946664 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.946629 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee9b04a4-387d-4702-b4ad-9a273642e191-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:58.952881 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.952301 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 18:35:58.953047 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.952915 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 18:35:58.953047 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.953012 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-znpc4\"" Oct 30 18:35:58.953131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.953088 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Oct 30 18:35:58.961262 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.961239 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Oct 30 18:35:58.961394 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:58.961354 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Oct 30 18:35:59.047871 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.047826 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.047871 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.047877 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.048179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.047900 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.048179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.047923 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ee9b04a4-387d-4702-b4ad-9a273642e191-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.048179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.047941 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee9b04a4-387d-4702-b4ad-9a273642e191-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.048179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.047997 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg5sq\" (UniqueName: \"kubernetes.io/projected/ee9b04a4-387d-4702-b4ad-9a273642e191-kube-api-access-cg5sq\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.048483 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.048448 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ee9b04a4-387d-4702-b4ad-9a273642e191-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.050528 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.050495 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee9b04a4-387d-4702-b4ad-9a273642e191-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.050669 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.050553 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.050734 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.050685 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.050734 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.050692 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee9b04a4-387d-4702-b4ad-9a273642e191-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.075693 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.075657 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg5sq\" (UniqueName: \"kubernetes.io/projected/ee9b04a4-387d-4702-b4ad-9a273642e191-kube-api-access-cg5sq\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-h244v\" (UID: \"ee9b04a4-387d-4702-b4ad-9a273642e191\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.197453 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.197367 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:35:59.569297 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:35:59.569168 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v"] Oct 30 18:35:59.573880 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:35:59.573851 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9b04a4_387d_4702_b4ad_9a273642e191.slice/crio-1dc0461022a8587f167b4f9268754d6d15ccbb2403986b62a4dfa599e5852746 WatchSource:0}: Error finding container 1dc0461022a8587f167b4f9268754d6d15ccbb2403986b62a4dfa599e5852746: Status 404 returned error can't find the container with id 1dc0461022a8587f167b4f9268754d6d15ccbb2403986b62a4dfa599e5852746 Oct 30 18:36:00.461520 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:00.461476 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" event={"ID":"ee9b04a4-387d-4702-b4ad-9a273642e191","Type":"ContainerStarted","Data":"1dc0461022a8587f167b4f9268754d6d15ccbb2403986b62a4dfa599e5852746"} Oct 30 18:36:02.470855 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:02.470807 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" event={"ID":"ee9b04a4-387d-4702-b4ad-9a273642e191","Type":"ContainerStarted","Data":"758fec811dbf0346e2204031c0744a14e229ae817f013129a52c555e72af15ec"} Oct 30 18:36:03.476434 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:03.476402 6160 generic.go:356] "Generic (PLEG): container finished" podID="ee9b04a4-387d-4702-b4ad-9a273642e191" containerID="758fec811dbf0346e2204031c0744a14e229ae817f013129a52c555e72af15ec" exitCode=0 Oct 30 18:36:03.476970 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:03.476484 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" event={"ID":"ee9b04a4-387d-4702-b4ad-9a273642e191","Type":"ContainerDied","Data":"758fec811dbf0346e2204031c0744a14e229ae817f013129a52c555e72af15ec"} Oct 30 18:36:04.482158 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:04.482119 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" event={"ID":"ee9b04a4-387d-4702-b4ad-9a273642e191","Type":"ContainerStarted","Data":"22237c0c8ae7b5e7e83deca4fe3e5cb7e2e84ff53cbe38c5da7a1c83c3eaccd1"} Oct 30 18:36:04.482649 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:04.482238 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:36:04.567584 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:04.567519 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" podStartSLOduration=3.008536995 podStartE2EDuration="6.56750599s" totalImagesPullingTime="3.137794426s" totalInitContainerRuntime="421.174569ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:04.555352776 +0000 UTC m=+235.093544975" watchObservedRunningTime="2026-10-30 18:36:04.56750599 +0000 UTC m=+235.105698195" Oct 30 18:36:14.394909 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:36:14.394870 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:36:15.489370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:15.489339 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-h244v" Oct 30 18:36:24.185014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.184981 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492"] Oct 30 18:36:24.187092 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.187072 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.208506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.208480 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Oct 30 18:36:24.213241 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.213220 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kmnd7\"" Oct 30 18:36:24.220495 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.220470 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Oct 30 18:36:24.220616 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.220523 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 18:36:24.220616 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.220546 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 18:36:24.221187 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.221163 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492"] Oct 30 18:36:24.257083 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.257062 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.257181 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.257100 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.257181 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.257122 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2vh\" (UniqueName: \"kubernetes.io/projected/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-kube-api-access-2q2vh\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.358039 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.358010 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.358214 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.358052 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.358214 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.358076 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2vh\" (UniqueName: \"kubernetes.io/projected/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-kube-api-access-2q2vh\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.358846 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.358821 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.360512 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.360477 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.380768 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.380744 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2vh\" (UniqueName: \"kubernetes.io/projected/fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e-kube-api-access-2q2vh\") pod \"kuadrant-console-plugin-66c4fb575d-rp492\" (UID: \"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.508829 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.508741 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" Oct 30 18:36:24.815186 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:24.815149 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492"] Oct 30 18:36:24.820085 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:36:24.820049 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe3f5e6e_42ae_4baa_bb2e_56f3bf15836e.slice/crio-92600a895d719545844e2b3421a3028c9e18f5fa6243777927504645e9649347 WatchSource:0}: Error finding container 92600a895d719545844e2b3421a3028c9e18f5fa6243777927504645e9649347: Status 404 returned error can't find the container with id 92600a895d719545844e2b3421a3028c9e18f5fa6243777927504645e9649347 Oct 30 18:36:25.559733 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:25.559686 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" event={"ID":"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e","Type":"ContainerStarted","Data":"92600a895d719545844e2b3421a3028c9e18f5fa6243777927504645e9649347"} Oct 30 18:36:28.066760 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.066719 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-869444d94f-84r4k"] Oct 30 18:36:28.091967 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.091933 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.094613 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.094591 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869444d94f-84r4k"] Oct 30 18:36:28.193546 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.193471 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Oct 30 18:36:28.193760 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.193697 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qx7lj\"" Oct 30 18:36:28.193833 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.193817 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Oct 30 18:36:28.196194 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.196171 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25tfn\" (UniqueName: \"kubernetes.io/projected/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-kube-api-access-25tfn\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.196324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.196213 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-tls-certs\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.196324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.196283 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-tmp\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.297409 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.297377 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-tmp\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.297573 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.297441 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25tfn\" (UniqueName: \"kubernetes.io/projected/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-kube-api-access-25tfn\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.297573 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.297471 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-tls-certs\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.299870 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.299846 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-tmp\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.300495 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.300470 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-tls-certs\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.355672 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.355598 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25tfn\" (UniqueName: \"kubernetes.io/projected/2860dd5f-fbe9-4e39-8910-e448f2d17cc0-kube-api-access-25tfn\") pod \"kube-auth-proxy-869444d94f-84r4k\" (UID: \"2860dd5f-fbe9-4e39-8910-e448f2d17cc0\") " pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.409069 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.409033 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" Oct 30 18:36:28.719333 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:28.719263 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869444d94f-84r4k"] Oct 30 18:36:28.724105 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:36:28.724077 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2860dd5f_fbe9_4e39_8910_e448f2d17cc0.slice/crio-f4a4de4114a446c64335db3593ec6ab664f34f1bdb0c185bde6f5ade81e81d19 WatchSource:0}: Error finding container f4a4de4114a446c64335db3593ec6ab664f34f1bdb0c185bde6f5ade81e81d19: Status 404 returned error can't find the container with id f4a4de4114a446c64335db3593ec6ab664f34f1bdb0c185bde6f5ade81e81d19 Oct 30 18:36:29.228712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.228682 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:36:29.232605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.232584 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.247513 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.247490 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 30 18:36:29.253584 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.253559 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:36:29.307370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.307336 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvrgb\" (UniqueName: \"kubernetes.io/projected/89b9a796-7be3-4993-a58a-545b4ffc1c0a-kube-api-access-jvrgb\") pod \"limitador-limitador-7c9b87d574-dtzmt\" (UID: \"89b9a796-7be3-4993-a58a-545b4ffc1c0a\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.307516 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.307431 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89b9a796-7be3-4993-a58a-545b4ffc1c0a-config-file\") pod \"limitador-limitador-7c9b87d574-dtzmt\" (UID: \"89b9a796-7be3-4993-a58a-545b4ffc1c0a\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.320762 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.320735 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:36:29.408417 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.408392 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89b9a796-7be3-4993-a58a-545b4ffc1c0a-config-file\") pod \"limitador-limitador-7c9b87d574-dtzmt\" (UID: \"89b9a796-7be3-4993-a58a-545b4ffc1c0a\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.408526 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.408471 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrgb\" (UniqueName: \"kubernetes.io/projected/89b9a796-7be3-4993-a58a-545b4ffc1c0a-kube-api-access-jvrgb\") pod \"limitador-limitador-7c9b87d574-dtzmt\" (UID: \"89b9a796-7be3-4993-a58a-545b4ffc1c0a\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.409166 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.409053 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89b9a796-7be3-4993-a58a-545b4ffc1c0a-config-file\") pod \"limitador-limitador-7c9b87d574-dtzmt\" (UID: \"89b9a796-7be3-4993-a58a-545b4ffc1c0a\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.453032 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.453003 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrgb\" (UniqueName: \"kubernetes.io/projected/89b9a796-7be3-4993-a58a-545b4ffc1c0a-kube-api-access-jvrgb\") pod \"limitador-limitador-7c9b87d574-dtzmt\" (UID: \"89b9a796-7be3-4993-a58a-545b4ffc1c0a\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.547971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.547909 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:29.591702 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.591540 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" event={"ID":"2860dd5f-fbe9-4e39-8910-e448f2d17cc0","Type":"ContainerStarted","Data":"f4a4de4114a446c64335db3593ec6ab664f34f1bdb0c185bde6f5ade81e81d19"} Oct 30 18:36:29.597133 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.596418 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" event={"ID":"fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e","Type":"ContainerStarted","Data":"088bca27dc80fd1f765eceb5f6ee270cd126c5ae4363526f671bd6f230a0aaa2"} Oct 30 18:36:29.856188 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.855782 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rp492" podStartSLOduration=1.347503765 podStartE2EDuration="5.8557335s" totalImagesPullingTime="4.508229735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:29.637670361 +0000 UTC m=+260.175862568" watchObservedRunningTime="2026-10-30 18:36:29.8557335 +0000 UTC m=+260.393925705" Oct 30 18:36:29.857745 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:29.857672 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:36:29.862203 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:36:29.862179 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b9a796_7be3_4993_a58a_545b4ffc1c0a.slice/crio-7e70215e8c4c5e48cf49ecf1ce86ccf3cd375ecd31a1507e04c35fd0c9151cc7 WatchSource:0}: Error finding container 7e70215e8c4c5e48cf49ecf1ce86ccf3cd375ecd31a1507e04c35fd0c9151cc7: Status 404 returned error can't find the container with id 7e70215e8c4c5e48cf49ecf1ce86ccf3cd375ecd31a1507e04c35fd0c9151cc7 Oct 30 18:36:30.602790 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:30.602737 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" event={"ID":"89b9a796-7be3-4993-a58a-545b4ffc1c0a","Type":"ContainerStarted","Data":"7e70215e8c4c5e48cf49ecf1ce86ccf3cd375ecd31a1507e04c35fd0c9151cc7"} Oct 30 18:36:31.609185 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:31.609146 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" event={"ID":"2860dd5f-fbe9-4e39-8910-e448f2d17cc0","Type":"ContainerStarted","Data":"00044437cf9b1e0a24c8659bbb9fc8b40d66269a8a50d80e6df17bf70fadef07"} Oct 30 18:36:31.664053 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:31.663945 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-869444d94f-84r4k" podStartSLOduration=1.1569735859999999 podStartE2EDuration="3.663929249s" totalImagesPullingTime="2.506955663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:31.663141272 +0000 UTC m=+262.201333476" watchObservedRunningTime="2026-10-30 18:36:31.663929249 +0000 UTC m=+262.202121454" Oct 30 18:36:32.614879 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:32.614844 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" event={"ID":"89b9a796-7be3-4993-a58a-545b4ffc1c0a","Type":"ContainerStarted","Data":"b6fe5418dcd0e07765e0eb0df8f05fec033d5617f285ccc9758063ceb8b97e64"} Oct 30 18:36:32.615295 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:32.615046 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:32.655183 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:32.655126 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" podStartSLOduration=1.022239332 podStartE2EDuration="3.655109752s" totalImagesPullingTime="2.63287042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:32.654032896 +0000 UTC m=+263.192225102" watchObservedRunningTime="2026-10-30 18:36:32.655109752 +0000 UTC m=+263.193301960" Oct 30 18:36:43.621263 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:36:43.621234 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-dtzmt" Oct 30 18:36:44.394417 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:36:44.394384 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:37:11.857443 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.857402 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c5b87b589-7fw2t"] Oct 30 18:37:11.878101 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.878066 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:11.886505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.886467 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5b87b589-7fw2t"] Oct 30 18:37:11.893105 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.893074 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 18:37:11.893283 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.893189 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 18:37:11.896940 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.896918 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 30 18:37:11.897438 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.897420 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4q6fz\"" Oct 30 18:37:11.897534 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.897520 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 30 18:37:11.897612 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.897589 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 30 18:37:11.969048 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.969015 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bbdb8-94a4-423d-91ef-3e7da516e56b-webhook-cert\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:11.969222 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.969067 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bbdb8-94a4-423d-91ef-3e7da516e56b-metrics-certs\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:11.969222 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:11.969163 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmn52\" (UniqueName: \"kubernetes.io/projected/486bbdb8-94a4-423d-91ef-3e7da516e56b-kube-api-access-qmn52\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.069605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.069568 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bbdb8-94a4-423d-91ef-3e7da516e56b-webhook-cert\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.069605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.069615 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bbdb8-94a4-423d-91ef-3e7da516e56b-metrics-certs\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.069797 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.069648 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmn52\" (UniqueName: \"kubernetes.io/projected/486bbdb8-94a4-423d-91ef-3e7da516e56b-kube-api-access-qmn52\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.072048 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.072016 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bbdb8-94a4-423d-91ef-3e7da516e56b-webhook-cert\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.072175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.072150 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bbdb8-94a4-423d-91ef-3e7da516e56b-metrics-certs\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.092749 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.092723 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmn52\" (UniqueName: \"kubernetes.io/projected/486bbdb8-94a4-423d-91ef-3e7da516e56b-kube-api-access-qmn52\") pod \"maas-controller-5c5b87b589-7fw2t\" (UID: \"486bbdb8-94a4-423d-91ef-3e7da516e56b\") " pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.191665 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.191586 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:12.501497 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.501417 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5b87b589-7fw2t"] Oct 30 18:37:12.507187 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:37:12.507165 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486bbdb8_94a4_423d_91ef_3e7da516e56b.slice/crio-2402ab3b64531c7ac76891866d670879a2c99f33bffdd23da09fe8f62eac4d4c WatchSource:0}: Error finding container 2402ab3b64531c7ac76891866d670879a2c99f33bffdd23da09fe8f62eac4d4c: Status 404 returned error can't find the container with id 2402ab3b64531c7ac76891866d670879a2c99f33bffdd23da09fe8f62eac4d4c Oct 30 18:37:12.772992 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:12.772904 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" event={"ID":"486bbdb8-94a4-423d-91ef-3e7da516e56b","Type":"ContainerStarted","Data":"2402ab3b64531c7ac76891866d670879a2c99f33bffdd23da09fe8f62eac4d4c"} Oct 30 18:37:14.397434 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:37:14.397392 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:37:14.783510 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:14.783476 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" event={"ID":"486bbdb8-94a4-423d-91ef-3e7da516e56b","Type":"ContainerStarted","Data":"f60db234e08d4a97ddeafd681729bf95650fbd032175de44f4746e3a052b1fd6"} Oct 30 18:37:14.783678 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:14.783539 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:14.825410 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:14.825353 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" podStartSLOduration=1.669349456 podStartE2EDuration="3.825337166s" totalImagesPullingTime="2.15598771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:14.824492175 +0000 UTC m=+305.362684381" watchObservedRunningTime="2026-10-30 18:37:14.825337166 +0000 UTC m=+305.363529367" Oct 30 18:37:15.970912 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:15.970877 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj"] Oct 30 18:37:15.974429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:15.974410 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:15.991596 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:15.991572 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 30 18:37:16.003471 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.003444 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 30 18:37:16.003471 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.003463 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 30 18:37:16.003599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.003490 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-85t6d\"" Oct 30 18:37:16.004372 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.004357 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj"] Oct 30 18:37:16.108032 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.107995 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fc434352-f592-4d62-b816-25a0786a696f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.108218 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.108049 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/fc434352-f592-4d62-b816-25a0786a696f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.108218 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.108129 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc434352-f592-4d62-b816-25a0786a696f-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.108344 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.108291 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98zp\" (UniqueName: \"kubernetes.io/projected/fc434352-f592-4d62-b816-25a0786a696f-kube-api-access-v98zp\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.209681 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.209648 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v98zp\" (UniqueName: \"kubernetes.io/projected/fc434352-f592-4d62-b816-25a0786a696f-kube-api-access-v98zp\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.209882 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.209692 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fc434352-f592-4d62-b816-25a0786a696f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.209882 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.209725 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/fc434352-f592-4d62-b816-25a0786a696f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.209882 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.209742 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc434352-f592-4d62-b816-25a0786a696f-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.210395 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.210373 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc434352-f592-4d62-b816-25a0786a696f-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.210677 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.210660 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/fc434352-f592-4d62-b816-25a0786a696f-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.212239 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.212219 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fc434352-f592-4d62-b816-25a0786a696f-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.237469 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.237399 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98zp\" (UniqueName: \"kubernetes.io/projected/fc434352-f592-4d62-b816-25a0786a696f-kube-api-access-v98zp\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-rxctj\" (UID: \"fc434352-f592-4d62-b816-25a0786a696f\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.286592 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.286556 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:16.586388 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.586360 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj"] Oct 30 18:37:16.590794 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:37:16.590771 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc434352_f592_4d62_b816_25a0786a696f.slice/crio-4313601b7614ac5396d34806bab1f3c3bd3c7cd1514ce3487cf9be55aca7ee81 WatchSource:0}: Error finding container 4313601b7614ac5396d34806bab1f3c3bd3c7cd1514ce3487cf9be55aca7ee81: Status 404 returned error can't find the container with id 4313601b7614ac5396d34806bab1f3c3bd3c7cd1514ce3487cf9be55aca7ee81 Oct 30 18:37:16.793077 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:16.793037 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" event={"ID":"fc434352-f592-4d62-b816-25a0786a696f","Type":"ContainerStarted","Data":"4313601b7614ac5396d34806bab1f3c3bd3c7cd1514ce3487cf9be55aca7ee81"} Oct 30 18:37:25.790377 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:25.790343 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:37:26.842449 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:26.842411 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" event={"ID":"fc434352-f592-4d62-b816-25a0786a696f","Type":"ContainerStarted","Data":"834180f6d30d462ee9011afd06cee6e77525ffe389f6f4ae27c9e56e86a12a61"} Oct 30 18:37:26.842827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:26.842498 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:26.915487 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:26.915441 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" podStartSLOduration=2.204863748 podStartE2EDuration="11.91542924s" totalImagesPullingTime="9.710565492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:26.90988752 +0000 UTC m=+317.448079723" watchObservedRunningTime="2026-10-30 18:37:26.91542924 +0000 UTC m=+317.453621456" Oct 30 18:37:34.706296 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.706263 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b"] Oct 30 18:37:34.710131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.710111 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.742368 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.742340 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b"] Oct 30 18:37:34.872024 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.871990 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08f0f8f-8ee7-4566-badb-cf1f043eb004-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.872024 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.872022 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.872244 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.872042 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.872244 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.872057 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx5nf\" (UniqueName: \"kubernetes.io/projected/d08f0f8f-8ee7-4566-badb-cf1f043eb004-kube-api-access-xx5nf\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.872244 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.872179 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.872244 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.872228 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.872392 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.872268 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973538 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973449 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08f0f8f-8ee7-4566-badb-cf1f043eb004-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973538 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973486 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973538 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973508 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973538 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973524 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx5nf\" (UniqueName: \"kubernetes.io/projected/d08f0f8f-8ee7-4566-badb-cf1f043eb004-kube-api-access-xx5nf\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973819 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973591 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973819 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973678 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.973819 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.973715 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.974534 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.974504 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.975994 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.975969 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.976079 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.976013 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08f0f8f-8ee7-4566-badb-cf1f043eb004-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.976079 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.976037 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.976149 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.976120 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d08f0f8f-8ee7-4566-badb-cf1f043eb004-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:34.997679 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:34.997649 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx5nf\" (UniqueName: \"kubernetes.io/projected/d08f0f8f-8ee7-4566-badb-cf1f043eb004-kube-api-access-xx5nf\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:35.003062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.003042 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08f0f8f-8ee7-4566-badb-cf1f043eb004-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nw46b\" (UID: \"d08f0f8f-8ee7-4566-badb-cf1f043eb004\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:35.022066 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.022034 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:35.354997 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.354888 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b"] Oct 30 18:37:35.359661 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:37:35.359635 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08f0f8f_8ee7_4566_badb_cf1f043eb004.slice/crio-1423f48c699154fb7b9f65d43ff6534fedf8ee82e202a6b24202d02da21c6a9a WatchSource:0}: Error finding container 1423f48c699154fb7b9f65d43ff6534fedf8ee82e202a6b24202d02da21c6a9a: Status 404 returned error can't find the container with id 1423f48c699154fb7b9f65d43ff6534fedf8ee82e202a6b24202d02da21c6a9a Oct 30 18:37:35.362652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.362285 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:37:35.362652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.362360 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:37:35.882415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.882305 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" event={"ID":"d08f0f8f-8ee7-4566-badb-cf1f043eb004","Type":"ContainerStarted","Data":"a8c758d24bf78e10bbf5e3687ac56dfb925bf306a7f8ce6380eae01d05cc16db"} Oct 30 18:37:35.882415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.882373 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" event={"ID":"d08f0f8f-8ee7-4566-badb-cf1f043eb004","Type":"ContainerStarted","Data":"1423f48c699154fb7b9f65d43ff6534fedf8ee82e202a6b24202d02da21c6a9a"} Oct 30 18:37:35.882970 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:35.882514 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:36.001494 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:36.001437 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" podStartSLOduration=2.001421147 podStartE2EDuration="2.001421147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:36.001307478 +0000 UTC m=+326.539499685" watchObservedRunningTime="2026-10-30 18:37:36.001421147 +0000 UTC m=+326.539613353" Oct 30 18:37:36.888615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:36.888582 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nw46b" Oct 30 18:37:37.027764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.027730 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz"] Oct 30 18:37:37.028010 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.027948 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" podUID="302d31fc-2367-40b4-9d85-a702077783ea" containerName="discovery" containerID="cri-o://6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31" gracePeriod=30 Oct 30 18:37:37.268664 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.268646 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:37:37.394106 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394082 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394237 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394158 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394237 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394211 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394308 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394258 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394308 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394282 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394589 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394317 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394589 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394369 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert\") pod \"302d31fc-2367-40b4-9d85-a702077783ea\" (UID: \"302d31fc-2367-40b4-9d85-a702077783ea\") " Oct 30 18:37:37.394725 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.394694 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:37:37.396799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.396728 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:37:37.396799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.396743 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:37:37.396979 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.396816 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:37:37.396979 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.396840 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:37:37.396979 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.396936 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:37:37.396979 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.396975 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk" pod "302d31fc-2367-40b4-9d85-a702077783ea" (UID: "302d31fc-2367-40b4-9d85-a702077783ea"). InnerVolumeSpecName "kube-api-access-7b6bk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:37:37.495928 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.495899 6160 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-kubeconfig\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.496072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.495931 6160 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-cacerts\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.496072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.495945 6160 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/302d31fc-2367-40b4-9d85-a702077783ea-local-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.496072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.495973 6160 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-dns-cert\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.496072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.495987 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b6bk\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-kube-api-access-7b6bk\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.496072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.495999 6160 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/302d31fc-2367-40b4-9d85-a702077783ea-istio-token\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.496072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.496013 6160 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/302d31fc-2367-40b4-9d85-a702077783ea-istio-csr-ca-configmap\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:37:37.852377 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.852344 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxctj" Oct 30 18:37:37.893657 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.893623 6160 generic.go:356] "Generic (PLEG): container finished" podID="302d31fc-2367-40b4-9d85-a702077783ea" containerID="6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31" exitCode=0 Oct 30 18:37:37.894497 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.894475 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" Oct 30 18:37:37.895171 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.894472 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" event={"ID":"302d31fc-2367-40b4-9d85-a702077783ea","Type":"ContainerDied","Data":"6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31"} Oct 30 18:37:37.895391 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.895367 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz" event={"ID":"302d31fc-2367-40b4-9d85-a702077783ea","Type":"ContainerDied","Data":"66188ba2752083b599e56b719f95f0adadd60797e6123ab1c8551061b5b90afc"} Oct 30 18:37:37.895509 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.895410 6160 scope.go:122] "RemoveContainer" containerID="6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31" Oct 30 18:37:37.905104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.905085 6160 scope.go:122] "RemoveContainer" containerID="6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31" Oct 30 18:37:37.905392 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:37:37.905370 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31\": container with ID starting with 6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31 not found: ID does not exist" containerID="6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31" Oct 30 18:37:37.905474 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.905404 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31"} err="failed to get container status \"6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31\": rpc error: code = NotFound desc = could not find container \"6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31\": container with ID starting with 6b4df815babec964e4f7ca5c69825687087426873b7d422bd0074e2c6585ef31 not found: ID does not exist" Oct 30 18:37:37.956809 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.956784 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz"] Oct 30 18:37:37.960316 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:37.960266 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-skcpz"] Oct 30 18:37:39.576806 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:39.576767 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="302d31fc-2367-40b4-9d85-a702077783ea" path="/var/lib/kubelet/pods/302d31fc-2367-40b4-9d85-a702077783ea/volumes" Oct 30 18:37:44.395091 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:37:44.395051 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:37:47.333082 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.333044 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79b474c645-76zq4"] Oct 30 18:37:47.333480 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.333455 6160 cpu_manager.go:414] "containerMap: removing container" podUID="302d31fc-2367-40b4-9d85-a702077783ea" containerName="discovery" Oct 30 18:37:47.333480 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.333471 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="302d31fc-2367-40b4-9d85-a702077783ea" containerName="discovery" Oct 30 18:37:47.333568 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.333556 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="302d31fc-2367-40b4-9d85-a702077783ea" containerName="discovery" Oct 30 18:37:47.337947 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.337931 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.366506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.366441 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:37:47.366506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.366436 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 30 18:37:47.366506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.366440 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 18:37:47.366506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.366444 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:37:47.367860 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.367835 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b474c645-76zq4"] Oct 30 18:37:47.383306 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.383283 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.383415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.383321 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.383504 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.383485 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.383563 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.383519 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnxz4\" (UniqueName: \"kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.485569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.484901 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.485569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.484948 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnxz4\" (UniqueName: \"kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.485569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.485034 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.485569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.485080 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.486920 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.486891 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.488411 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.488375 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.488505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.488431 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.512135 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.512107 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnxz4\" (UniqueName: \"kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4\") pod \"maas-api-79b474c645-76zq4\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.650560 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.650474 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:47.948368 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:47.948275 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b474c645-76zq4"] Oct 30 18:37:47.952527 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:37:47.952499 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ede4a7_8db8_41a6_b653_5b36bb4c43c2.slice/crio-b0c4e414904b4832578772e9553f6448b9167f2e8a0f678d699e0354f43134e5 WatchSource:0}: Error finding container b0c4e414904b4832578772e9553f6448b9167f2e8a0f678d699e0354f43134e5: Status 404 returned error can't find the container with id b0c4e414904b4832578772e9553f6448b9167f2e8a0f678d699e0354f43134e5 Oct 30 18:37:48.939064 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:48.939024 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" event={"ID":"51ede4a7-8db8-41a6-b653-5b36bb4c43c2","Type":"ContainerStarted","Data":"b0c4e414904b4832578772e9553f6448b9167f2e8a0f678d699e0354f43134e5"} Oct 30 18:37:49.347044 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:49.347004 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:37:49.944763 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:49.944721 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" event={"ID":"51ede4a7-8db8-41a6-b653-5b36bb4c43c2","Type":"ContainerStarted","Data":"fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148"} Oct 30 18:37:49.945152 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:49.944830 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:49.986145 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:49.986095 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" podStartSLOduration=1.468809127 podStartE2EDuration="2.986082866s" totalImagesPullingTime="1.517273739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:49.985515743 +0000 UTC m=+340.523707949" watchObservedRunningTime="2026-10-30 18:37:49.986082866 +0000 UTC m=+340.524275095" Oct 30 18:37:55.954121 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:55.954081 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:37:58.280129 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.280092 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp"] Oct 30 18:37:58.551815 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.551730 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp"] Oct 30 18:37:58.551971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.551842 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.578176 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.578150 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bnjrn\"" Oct 30 18:37:58.679947 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.679915 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4b590be-8d71-4555-a7ae-d86b8e354fcd-metrics-certs\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.680123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.679949 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b590be-8d71-4555-a7ae-d86b8e354fcd-openshift-service-ca\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.680123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.680050 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4b590be-8d71-4555-a7ae-d86b8e354fcd-maas-api-tls\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.680197 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.680130 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6pfp\" (UniqueName: \"kubernetes.io/projected/c4b590be-8d71-4555-a7ae-d86b8e354fcd-kube-api-access-h6pfp\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.781063 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.781027 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4b590be-8d71-4555-a7ae-d86b8e354fcd-metrics-certs\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.781063 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.781065 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b590be-8d71-4555-a7ae-d86b8e354fcd-openshift-service-ca\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.781305 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.781090 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4b590be-8d71-4555-a7ae-d86b8e354fcd-maas-api-tls\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.781305 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.781135 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6pfp\" (UniqueName: \"kubernetes.io/projected/c4b590be-8d71-4555-a7ae-d86b8e354fcd-kube-api-access-h6pfp\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.781854 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.781834 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4b590be-8d71-4555-a7ae-d86b8e354fcd-openshift-service-ca\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.783476 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.783455 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4b590be-8d71-4555-a7ae-d86b8e354fcd-maas-api-tls\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.783545 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.783514 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4b590be-8d71-4555-a7ae-d86b8e354fcd-metrics-certs\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.804773 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.804696 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6pfp\" (UniqueName: \"kubernetes.io/projected/c4b590be-8d71-4555-a7ae-d86b8e354fcd-kube-api-access-h6pfp\") pod \"maas-api-6f8d6bc878-r4jhp\" (UID: \"c4b590be-8d71-4555-a7ae-d86b8e354fcd\") " pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:58.882535 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:58.882498 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:37:59.208872 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:59.208793 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp"] Oct 30 18:37:59.213550 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:37:59.213521 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b590be_8d71_4555_a7ae_d86b8e354fcd.slice/crio-19747e9506ee96830852e19bf7172439cc20ff2d3e9207c26d8eb4f3c84288ea WatchSource:0}: Error finding container 19747e9506ee96830852e19bf7172439cc20ff2d3e9207c26d8eb4f3c84288ea: Status 404 returned error can't find the container with id 19747e9506ee96830852e19bf7172439cc20ff2d3e9207c26d8eb4f3c84288ea Oct 30 18:37:59.989486 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:59.989410 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" event={"ID":"c4b590be-8d71-4555-a7ae-d86b8e354fcd","Type":"ContainerStarted","Data":"32da08ceb36a9fdbfc1bf6eca361e6b3168fb0840d7a106c6223ac5ae57f9df4"} Oct 30 18:37:59.989486 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:59.989469 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" event={"ID":"c4b590be-8d71-4555-a7ae-d86b8e354fcd","Type":"ContainerStarted","Data":"19747e9506ee96830852e19bf7172439cc20ff2d3e9207c26d8eb4f3c84288ea"} Oct 30 18:37:59.989875 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:37:59.989509 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:38:00.035416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:00.035367 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" podStartSLOduration=1.6170160139999998 podStartE2EDuration="2.035352229s" totalImagesPullingTime="418.336215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:00.034046095 +0000 UTC m=+350.572238314" watchObservedRunningTime="2026-10-30 18:38:00.035352229 +0000 UTC m=+350.573544428" Oct 30 18:38:05.998445 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:05.998413 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6f8d6bc878-r4jhp" Oct 30 18:38:06.075868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.075833 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b474c645-76zq4"] Oct 30 18:38:06.076149 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.076116 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" podUID="51ede4a7-8db8-41a6-b653-5b36bb4c43c2" containerName="maas-api" containerID="cri-o://fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148" gracePeriod=30 Oct 30 18:38:06.318276 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.318252 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:38:06.449721 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.449684 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca\") pod \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " Oct 30 18:38:06.449895 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.449731 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs\") pod \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " Oct 30 18:38:06.449895 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.449814 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4\" (UniqueName: \"kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4\") pod \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " Oct 30 18:38:06.449895 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.449846 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls\") pod \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\" (UID: \"51ede4a7-8db8-41a6-b653-5b36bb4c43c2\") " Oct 30 18:38:06.450166 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.450134 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca" pod "51ede4a7-8db8-41a6-b653-5b36bb4c43c2" (UID: "51ede4a7-8db8-41a6-b653-5b36bb4c43c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:06.452047 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.452017 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls" pod "51ede4a7-8db8-41a6-b653-5b36bb4c43c2" (UID: "51ede4a7-8db8-41a6-b653-5b36bb4c43c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:06.452155 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.452040 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs" pod "51ede4a7-8db8-41a6-b653-5b36bb4c43c2" (UID: "51ede4a7-8db8-41a6-b653-5b36bb4c43c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:06.452155 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.452138 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4" pod "51ede4a7-8db8-41a6-b653-5b36bb4c43c2" (UID: "51ede4a7-8db8-41a6-b653-5b36bb4c43c2"). InnerVolumeSpecName "kube-api-access-gnxz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:06.551114 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.551034 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnxz4\" (UniqueName: \"kubernetes.io/projected/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-kube-api-access-gnxz4\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:38:06.551114 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.551064 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:38:06.551114 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.551073 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:38:06.551114 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:06.551082 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51ede4a7-8db8-41a6-b653-5b36bb4c43c2-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:38:07.019722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.019634 6160 generic.go:356] "Generic (PLEG): container finished" podID="51ede4a7-8db8-41a6-b653-5b36bb4c43c2" containerID="fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148" exitCode=0 Oct 30 18:38:07.019722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.019684 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" Oct 30 18:38:07.020238 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.019683 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" event={"ID":"51ede4a7-8db8-41a6-b653-5b36bb4c43c2","Type":"ContainerDied","Data":"fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148"} Oct 30 18:38:07.020238 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.019834 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79b474c645-76zq4" event={"ID":"51ede4a7-8db8-41a6-b653-5b36bb4c43c2","Type":"ContainerDied","Data":"b0c4e414904b4832578772e9553f6448b9167f2e8a0f678d699e0354f43134e5"} Oct 30 18:38:07.020238 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.019875 6160 scope.go:122] "RemoveContainer" containerID="fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148" Oct 30 18:38:07.029310 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.029290 6160 scope.go:122] "RemoveContainer" containerID="fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148" Oct 30 18:38:07.029595 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:38:07.029574 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148\": container with ID starting with fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148 not found: ID does not exist" containerID="fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148" Oct 30 18:38:07.029652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.029604 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148"} err="failed to get container status \"fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148\": rpc error: code = NotFound desc = could not find container \"fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148\": container with ID starting with fae755c1369320d8a25766c9aa956ff90e69c1c2a869159a59f8ef851e340148 not found: ID does not exist" Oct 30 18:38:07.069597 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.069564 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b474c645-76zq4"] Oct 30 18:38:07.081493 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.081466 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-79b474c645-76zq4"] Oct 30 18:38:07.575928 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:38:07.575893 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51ede4a7-8db8-41a6-b653-5b36bb4c43c2" path="/var/lib/kubelet/pods/51ede4a7-8db8-41a6-b653-5b36bb4c43c2/volumes" Oct 30 18:38:14.395637 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:38:14.395600 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:38:44.395153 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:38:44.395116 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:39:14.395266 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:39:14.395224 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:39:16.098943 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.098909 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-qd9ff"] Oct 30 18:39:16.099480 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.099459 6160 cpu_manager.go:414] "containerMap: removing container" podUID="51ede4a7-8db8-41a6-b653-5b36bb4c43c2" containerName="maas-api" Oct 30 18:39:16.103601 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.099483 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51ede4a7-8db8-41a6-b653-5b36bb4c43c2" containerName="maas-api" Oct 30 18:39:16.103601 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.099570 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="51ede4a7-8db8-41a6-b653-5b36bb4c43c2" containerName="maas-api" Oct 30 18:39:16.103601 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.102467 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.118592 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.118566 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" Oct 30 18:39:16.118737 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.118721 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Oct 30 18:39:16.130442 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.130422 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-nrq2j\"" Oct 30 18:39:16.131390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.131370 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-qd9ff"] Oct 30 18:39:16.227369 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.227339 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmppj\" (UniqueName: \"kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj\") pod \"ai-gateway-controller-7d85675f98-qd9ff\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.227521 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.227384 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-qd9ff\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.328612 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.328572 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmppj\" (UniqueName: \"kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj\") pod \"ai-gateway-controller-7d85675f98-qd9ff\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.328612 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.328615 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-qd9ff\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.330934 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.330915 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-qd9ff\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.352532 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.352460 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmppj\" (UniqueName: \"kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj\") pod \"ai-gateway-controller-7d85675f98-qd9ff\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.417013 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.416982 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:16.712687 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:16.712598 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-qd9ff"] Oct 30 18:39:16.717768 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:39:16.717738 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897d4ce2_9fbe_400a_9ead_461291cb42b7.slice/crio-6b4c66327108cea9b20bf69df6685a2ba3833ae5c1e1a305f659e921811bdd73 WatchSource:0}: Error finding container 6b4c66327108cea9b20bf69df6685a2ba3833ae5c1e1a305f659e921811bdd73: Status 404 returned error can't find the container with id 6b4c66327108cea9b20bf69df6685a2ba3833ae5c1e1a305f659e921811bdd73 Oct 30 18:39:17.314967 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:17.314922 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" event={"ID":"897d4ce2-9fbe-400a-9ead-461291cb42b7","Type":"ContainerStarted","Data":"6b4c66327108cea9b20bf69df6685a2ba3833ae5c1e1a305f659e921811bdd73"} Oct 30 18:39:18.321627 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:18.321592 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" event={"ID":"897d4ce2-9fbe-400a-9ead-461291cb42b7","Type":"ContainerStarted","Data":"8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a"} Oct 30 18:39:18.322160 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:18.321696 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:18.374402 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:18.374348 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" podStartSLOduration=0.917948863 podStartE2EDuration="2.374332011s" totalImagesPullingTime="1.456383148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:18.36201158 +0000 UTC m=+428.900203785" watchObservedRunningTime="2026-10-30 18:39:18.374332011 +0000 UTC m=+428.912524216" Oct 30 18:39:20.727018 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.726967 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx"] Oct 30 18:39:20.730524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.730505 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:20.746796 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.746774 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Oct 30 18:39:20.758104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.758075 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-frcwd\"" Oct 30 18:39:20.759165 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.759134 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx"] Oct 30 18:39:20.782887 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.782853 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-b8qh5"] Oct 30 18:39:20.786370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.786349 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:20.814240 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.814207 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-b8qh5"] Oct 30 18:39:20.874588 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.874556 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcwtj\" (UniqueName: \"kubernetes.io/projected/720c779d-e253-4480-b411-75e6ca6e3dbc-kube-api-access-hcwtj\") pod \"payload-processing-5f78687d5b-b8qh5\" (UID: \"720c779d-e253-4480-b411-75e6ca6e3dbc\") " pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:20.874758 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.874650 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/720c779d-e253-4480-b411-75e6ca6e3dbc-plugins-config-volume\") pod \"payload-processing-5f78687d5b-b8qh5\" (UID: \"720c779d-e253-4480-b411-75e6ca6e3dbc\") " pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:20.874758 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.874723 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw49n\" (UniqueName: \"kubernetes.io/projected/44a75941-102f-406b-8e35-d43c8fb93408-kube-api-access-nw49n\") pod \"payload-pre-processing-7b98bb678c-n2ljx\" (UID: \"44a75941-102f-406b-8e35-d43c8fb93408\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:20.874862 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.874757 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a75941-102f-406b-8e35-d43c8fb93408-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-n2ljx\" (UID: \"44a75941-102f-406b-8e35-d43c8fb93408\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:20.975724 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.975691 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/720c779d-e253-4480-b411-75e6ca6e3dbc-plugins-config-volume\") pod \"payload-processing-5f78687d5b-b8qh5\" (UID: \"720c779d-e253-4480-b411-75e6ca6e3dbc\") " pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:20.975923 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.975775 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw49n\" (UniqueName: \"kubernetes.io/projected/44a75941-102f-406b-8e35-d43c8fb93408-kube-api-access-nw49n\") pod \"payload-pre-processing-7b98bb678c-n2ljx\" (UID: \"44a75941-102f-406b-8e35-d43c8fb93408\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:20.975923 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.975823 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a75941-102f-406b-8e35-d43c8fb93408-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-n2ljx\" (UID: \"44a75941-102f-406b-8e35-d43c8fb93408\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:20.975923 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.975873 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcwtj\" (UniqueName: \"kubernetes.io/projected/720c779d-e253-4480-b411-75e6ca6e3dbc-kube-api-access-hcwtj\") pod \"payload-processing-5f78687d5b-b8qh5\" (UID: \"720c779d-e253-4480-b411-75e6ca6e3dbc\") " pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:20.976533 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.976515 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/720c779d-e253-4480-b411-75e6ca6e3dbc-plugins-config-volume\") pod \"payload-processing-5f78687d5b-b8qh5\" (UID: \"720c779d-e253-4480-b411-75e6ca6e3dbc\") " pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:20.976577 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:20.976559 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a75941-102f-406b-8e35-d43c8fb93408-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-n2ljx\" (UID: \"44a75941-102f-406b-8e35-d43c8fb93408\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:21.000209 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:21.000147 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcwtj\" (UniqueName: \"kubernetes.io/projected/720c779d-e253-4480-b411-75e6ca6e3dbc-kube-api-access-hcwtj\") pod \"payload-processing-5f78687d5b-b8qh5\" (UID: \"720c779d-e253-4480-b411-75e6ca6e3dbc\") " pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:21.000209 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:21.000152 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw49n\" (UniqueName: \"kubernetes.io/projected/44a75941-102f-406b-8e35-d43c8fb93408-kube-api-access-nw49n\") pod \"payload-pre-processing-7b98bb678c-n2ljx\" (UID: \"44a75941-102f-406b-8e35-d43c8fb93408\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:21.043235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:21.043210 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:21.100034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:21.100001 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:21.343586 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:21.343548 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx"] Oct 30 18:39:21.444335 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:21.444298 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-b8qh5"] Oct 30 18:39:21.447879 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:39:21.447853 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720c779d_e253_4480_b411_75e6ca6e3dbc.slice/crio-bcee6b6a28d29548da5bdd505d8b5fdf99e9db920d91160039a5cca847f3c4ca WatchSource:0}: Error finding container bcee6b6a28d29548da5bdd505d8b5fdf99e9db920d91160039a5cca847f3c4ca: Status 404 returned error can't find the container with id bcee6b6a28d29548da5bdd505d8b5fdf99e9db920d91160039a5cca847f3c4ca Oct 30 18:39:22.340505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:22.340468 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" event={"ID":"720c779d-e253-4480-b411-75e6ca6e3dbc","Type":"ContainerStarted","Data":"bcee6b6a28d29548da5bdd505d8b5fdf99e9db920d91160039a5cca847f3c4ca"} Oct 30 18:39:22.341792 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:22.341759 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" event={"ID":"44a75941-102f-406b-8e35-d43c8fb93408","Type":"ContainerStarted","Data":"9a6039a748ffde4124e5b6578fade08c551f39bac2b9fb599d3f39bae40430ea"} Oct 30 18:39:24.355673 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:24.355632 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" event={"ID":"720c779d-e253-4480-b411-75e6ca6e3dbc","Type":"ContainerStarted","Data":"0cedf6191083b812e3057cde291168d8b4942e539992f6b458215a0d93b2297c"} Oct 30 18:39:24.356194 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:24.355725 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:24.357312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:24.357284 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" event={"ID":"44a75941-102f-406b-8e35-d43c8fb93408","Type":"ContainerStarted","Data":"99289a6ddde153edda74ed1a03e71b6037a8b7660b1f6a459b6fec22b0e1229d"} Oct 30 18:39:24.357442 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:24.357336 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:24.396457 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:24.396406 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" podStartSLOduration=2.230853524 podStartE2EDuration="4.39638836s" totalImagesPullingTime="2.165534836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:24.395245451 +0000 UTC m=+434.933437655" watchObservedRunningTime="2026-10-30 18:39:24.39638836 +0000 UTC m=+434.934580568" Oct 30 18:39:24.437189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:24.437124 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" podStartSLOduration=2.140999303 podStartE2EDuration="4.437111214s" totalImagesPullingTime="2.296111911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:24.436668585 +0000 UTC m=+434.974860790" watchObservedRunningTime="2026-10-30 18:39:24.437111214 +0000 UTC m=+434.975303418" Oct 30 18:39:29.328773 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.328741 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:29.616156 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.616044 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7b4cb8955-jcvnm"] Oct 30 18:39:29.619684 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.619662 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.639942 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.639916 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7b4cb8955-jcvnm"] Oct 30 18:39:29.749390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.749357 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d43b61-fbe2-4332-beac-45f26bf72788-metrics-certs\") pod \"ai-gateway-controller-7b4cb8955-jcvnm\" (UID: \"59d43b61-fbe2-4332-beac-45f26bf72788\") " pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.749539 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.749478 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm642\" (UniqueName: \"kubernetes.io/projected/59d43b61-fbe2-4332-beac-45f26bf72788-kube-api-access-tm642\") pod \"ai-gateway-controller-7b4cb8955-jcvnm\" (UID: \"59d43b61-fbe2-4332-beac-45f26bf72788\") " pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.849848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.849821 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm642\" (UniqueName: \"kubernetes.io/projected/59d43b61-fbe2-4332-beac-45f26bf72788-kube-api-access-tm642\") pod \"ai-gateway-controller-7b4cb8955-jcvnm\" (UID: \"59d43b61-fbe2-4332-beac-45f26bf72788\") " pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.850006 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.849869 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d43b61-fbe2-4332-beac-45f26bf72788-metrics-certs\") pod \"ai-gateway-controller-7b4cb8955-jcvnm\" (UID: \"59d43b61-fbe2-4332-beac-45f26bf72788\") " pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.852290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.852273 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59d43b61-fbe2-4332-beac-45f26bf72788-metrics-certs\") pod \"ai-gateway-controller-7b4cb8955-jcvnm\" (UID: \"59d43b61-fbe2-4332-beac-45f26bf72788\") " pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.873422 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.873359 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm642\" (UniqueName: \"kubernetes.io/projected/59d43b61-fbe2-4332-beac-45f26bf72788-kube-api-access-tm642\") pod \"ai-gateway-controller-7b4cb8955-jcvnm\" (UID: \"59d43b61-fbe2-4332-beac-45f26bf72788\") " pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:29.931783 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:29.931746 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:30.227281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:30.227191 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7b4cb8955-jcvnm"] Oct 30 18:39:30.232294 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:39:30.232264 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d43b61_fbe2_4332_beac_45f26bf72788.slice/crio-1331d6d8da52c4629131fecf682da2a834005ad49d60fbc5832501fe6c99ab1c WatchSource:0}: Error finding container 1331d6d8da52c4629131fecf682da2a834005ad49d60fbc5832501fe6c99ab1c: Status 404 returned error can't find the container with id 1331d6d8da52c4629131fecf682da2a834005ad49d60fbc5832501fe6c99ab1c Oct 30 18:39:30.383197 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:30.383153 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" event={"ID":"59d43b61-fbe2-4332-beac-45f26bf72788","Type":"ContainerStarted","Data":"1331d6d8da52c4629131fecf682da2a834005ad49d60fbc5832501fe6c99ab1c"} Oct 30 18:39:31.389838 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:31.389754 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" event={"ID":"59d43b61-fbe2-4332-beac-45f26bf72788","Type":"ContainerStarted","Data":"37e64cea321dffb30655c0c7dc58501c0a56622df0aa113d0181db0082d6ff11"} Oct 30 18:39:31.390223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:31.389877 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:31.431710 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:31.431649 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" podStartSLOduration=2.096923912 podStartE2EDuration="2.431633313s" totalImagesPullingTime="334.709401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:31.430583095 +0000 UTC m=+441.968775301" watchObservedRunningTime="2026-10-30 18:39:31.431633313 +0000 UTC m=+441.969825518" Oct 30 18:39:35.364808 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:35.364779 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-n2ljx" Oct 30 18:39:35.365232 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:35.364859 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-b8qh5" Oct 30 18:39:42.396529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.396497 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7b4cb8955-jcvnm" Oct 30 18:39:42.470880 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.470847 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-qd9ff"] Oct 30 18:39:42.471136 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.471100 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" podUID="897d4ce2-9fbe-400a-9ead-461291cb42b7" containerName="manager" containerID="cri-o://8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a" gracePeriod=30 Oct 30 18:39:42.713286 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.713258 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:42.749854 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.749819 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj\" (UniqueName: \"kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj\") pod \"897d4ce2-9fbe-400a-9ead-461291cb42b7\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " Oct 30 18:39:42.750045 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.749926 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs\") pod \"897d4ce2-9fbe-400a-9ead-461291cb42b7\" (UID: \"897d4ce2-9fbe-400a-9ead-461291cb42b7\") " Oct 30 18:39:42.751947 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.751910 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs" pod "897d4ce2-9fbe-400a-9ead-461291cb42b7" (UID: "897d4ce2-9fbe-400a-9ead-461291cb42b7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:39:42.752071 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.751983 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj" pod "897d4ce2-9fbe-400a-9ead-461291cb42b7" (UID: "897d4ce2-9fbe-400a-9ead-461291cb42b7"). InnerVolumeSpecName "kube-api-access-wmppj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:42.850910 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.850868 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmppj\" (UniqueName: \"kubernetes.io/projected/897d4ce2-9fbe-400a-9ead-461291cb42b7-kube-api-access-wmppj\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:39:42.850910 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:42.850904 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/897d4ce2-9fbe-400a-9ead-461291cb42b7-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:39:43.444670 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.444633 6160 generic.go:356] "Generic (PLEG): container finished" podID="897d4ce2-9fbe-400a-9ead-461291cb42b7" containerID="8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a" exitCode=0 Oct 30 18:39:43.445191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.444688 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" Oct 30 18:39:43.445191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.444725 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" event={"ID":"897d4ce2-9fbe-400a-9ead-461291cb42b7","Type":"ContainerDied","Data":"8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a"} Oct 30 18:39:43.445191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.444780 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-qd9ff" event={"ID":"897d4ce2-9fbe-400a-9ead-461291cb42b7","Type":"ContainerDied","Data":"6b4c66327108cea9b20bf69df6685a2ba3833ae5c1e1a305f659e921811bdd73"} Oct 30 18:39:43.445191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.444805 6160 scope.go:122] "RemoveContainer" containerID="8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a" Oct 30 18:39:43.454581 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.454566 6160 scope.go:122] "RemoveContainer" containerID="8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a" Oct 30 18:39:43.454852 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:39:43.454834 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a\": container with ID starting with 8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a not found: ID does not exist" containerID="8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a" Oct 30 18:39:43.454910 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.454858 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a"} err="failed to get container status \"8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a\": rpc error: code = NotFound desc = could not find container \"8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a\": container with ID starting with 8f6c5c5031c53d05d5881b92f3a5c517c1205c76f2708c712c3a27b02249fe5a not found: ID does not exist" Oct 30 18:39:43.494679 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.494647 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-qd9ff"] Oct 30 18:39:43.506295 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.506260 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-qd9ff"] Oct 30 18:39:43.576933 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:43.576896 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="897d4ce2-9fbe-400a-9ead-461291cb42b7" path="/var/lib/kubelet/pods/897d4ce2-9fbe-400a-9ead-461291cb42b7/volumes" Oct 30 18:39:44.395689 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:39:44.395653 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:39:44.451236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:44.451205 6160 generic.go:356] "Generic (PLEG): container finished" podID="486bbdb8-94a4-423d-91ef-3e7da516e56b" containerID="f60db234e08d4a97ddeafd681729bf95650fbd032175de44f4746e3a052b1fd6" exitCode=1 Oct 30 18:39:44.451621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:44.451253 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" event={"ID":"486bbdb8-94a4-423d-91ef-3e7da516e56b","Type":"ContainerDied","Data":"f60db234e08d4a97ddeafd681729bf95650fbd032175de44f4746e3a052b1fd6"} Oct 30 18:39:44.451621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:44.451605 6160 scope.go:122] "RemoveContainer" containerID="f60db234e08d4a97ddeafd681729bf95650fbd032175de44f4746e3a052b1fd6" Oct 30 18:39:45.457857 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:45.457824 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" event={"ID":"486bbdb8-94a4-423d-91ef-3e7da516e56b","Type":"ContainerStarted","Data":"af19a074dd1edf410d9717f74217a107130178fc6166e185f35eaefdcccaa3c2"} Oct 30 18:39:45.458367 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:45.458042 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:39:56.464661 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:39:56.464629 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c5b87b589-7fw2t" Oct 30 18:40:00.934550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.934510 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm"] Oct 30 18:40:00.940897 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.939079 6160 cpu_manager.go:414] "containerMap: removing container" podUID="897d4ce2-9fbe-400a-9ead-461291cb42b7" containerName="manager" Oct 30 18:40:00.940897 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.939104 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="897d4ce2-9fbe-400a-9ead-461291cb42b7" containerName="manager" Oct 30 18:40:00.940897 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.939320 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="897d4ce2-9fbe-400a-9ead-461291cb42b7" containerName="manager" Oct 30 18:40:00.943256 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.943231 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:00.957742 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.957714 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Oct 30 18:40:00.957858 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.957756 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Oct 30 18:40:00.969430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.969406 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qk5lt\"" Oct 30 18:40:00.969430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.969414 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Oct 30 18:40:00.970436 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:00.970417 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm"] Oct 30 18:40:01.011338 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.011316 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.011444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.011349 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2775d01f-942d-4bc1-a427-68b71d46324f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.011444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.011377 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.011444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.011427 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.011556 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.011452 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.011589 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.011579 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4h9t\" (UniqueName: \"kubernetes.io/projected/2775d01f-942d-4bc1-a427-68b71d46324f-kube-api-access-n4h9t\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.112777 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.112742 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.112904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.112782 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2775d01f-942d-4bc1-a427-68b71d46324f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.112904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.112813 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.112904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.112837 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.112904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.112853 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.113179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.112915 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4h9t\" (UniqueName: \"kubernetes.io/projected/2775d01f-942d-4bc1-a427-68b71d46324f-kube-api-access-n4h9t\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.113324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.113302 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.113421 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.113399 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.113854 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.113828 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.115606 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.115581 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2775d01f-942d-4bc1-a427-68b71d46324f-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.115870 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.115850 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2775d01f-942d-4bc1-a427-68b71d46324f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.135755 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.135732 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4h9t\" (UniqueName: \"kubernetes.io/projected/2775d01f-942d-4bc1-a427-68b71d46324f-kube-api-access-n4h9t\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm\" (UID: \"2775d01f-942d-4bc1-a427-68b71d46324f\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.256027 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.255974 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:01.560181 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:01.560143 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm"] Oct 30 18:40:01.565656 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:40:01.565628 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2775d01f_942d_4bc1_a427_68b71d46324f.slice/crio-58c0b3ab9cdcbd9f53eac3aeb2252f13108c4c4f3b155a61c73d734273523f03 WatchSource:0}: Error finding container 58c0b3ab9cdcbd9f53eac3aeb2252f13108c4c4f3b155a61c73d734273523f03: Status 404 returned error can't find the container with id 58c0b3ab9cdcbd9f53eac3aeb2252f13108c4c4f3b155a61c73d734273523f03 Oct 30 18:40:02.002881 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:02.002792 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:40:02.528699 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:02.528660 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" event={"ID":"2775d01f-942d-4bc1-a427-68b71d46324f","Type":"ContainerStarted","Data":"58c0b3ab9cdcbd9f53eac3aeb2252f13108c4c4f3b155a61c73d734273523f03"} Oct 30 18:40:03.533891 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:03.533851 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" event={"ID":"2775d01f-942d-4bc1-a427-68b71d46324f","Type":"ContainerStarted","Data":"7312a910ecdaac158568b70595acc8f9f15cd7da4f4b13e7a6719608abe5dd7e"} Oct 30 18:40:03.575651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:03.575603 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" podStartSLOduration=2.381345321 podStartE2EDuration="3.575590745s" totalImagesPullingTime="1.194245424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:03.574921591 +0000 UTC m=+474.113113797" watchObservedRunningTime="2026-10-30 18:40:03.575590745 +0000 UTC m=+474.113782944" Oct 30 18:40:04.256581 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:04.256544 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:07.411687 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:07.411651 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:40:10.679470 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:10.679429 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:40:11.256895 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:11.256863 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:11.269714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:11.269689 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:11.578495 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:11.578466 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm" Oct 30 18:40:11.972873 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:11.972801 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz"] Oct 30 18:40:11.978267 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:11.978244 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:11.992712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:11.992688 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Oct 30 18:40:12.000851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.000828 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz"] Oct 30 18:40:12.112645 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.112611 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.112811 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.112651 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjdqw\" (UniqueName: \"kubernetes.io/projected/2c95c7f7-7920-4be6-96b4-b93b17a36c68-kube-api-access-qjdqw\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.112811 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.112729 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c95c7f7-7920-4be6-96b4-b93b17a36c68-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.112887 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.112806 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.112887 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.112863 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.112949 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.112888 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214087 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214126 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjdqw\" (UniqueName: \"kubernetes.io/projected/2c95c7f7-7920-4be6-96b4-b93b17a36c68-kube-api-access-qjdqw\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214153 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c95c7f7-7920-4be6-96b4-b93b17a36c68-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214176 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214294 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214338 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214642 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214618 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214717 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214627 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.214717 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.214703 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.216546 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.216520 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c95c7f7-7920-4be6-96b4-b93b17a36c68-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.216808 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.216785 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c95c7f7-7920-4be6-96b4-b93b17a36c68-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.237040 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.236986 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjdqw\" (UniqueName: \"kubernetes.io/projected/2c95c7f7-7920-4be6-96b4-b93b17a36c68-kube-api-access-qjdqw\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xvbrz\" (UID: \"2c95c7f7-7920-4be6-96b4-b93b17a36c68\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.290897 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.290855 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:12.584475 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:12.584441 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz"] Oct 30 18:40:12.588576 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:40:12.588551 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c95c7f7_7920_4be6_96b4_b93b17a36c68.slice/crio-3ff50bac7c92c041e24a424ebcb42d7d4acdd13d38d1dbfe2c607827ca704bd4 WatchSource:0}: Error finding container 3ff50bac7c92c041e24a424ebcb42d7d4acdd13d38d1dbfe2c607827ca704bd4: Status 404 returned error can't find the container with id 3ff50bac7c92c041e24a424ebcb42d7d4acdd13d38d1dbfe2c607827ca704bd4 Oct 30 18:40:13.427517 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.427483 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4"] Oct 30 18:40:13.431607 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.431587 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.448219 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.448199 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Oct 30 18:40:13.461255 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.461228 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4"] Oct 30 18:40:13.526324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.526293 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.526569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.526332 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgf7g\" (UniqueName: \"kubernetes.io/projected/08fa115b-8e06-4399-ab65-657c78368182-kube-api-access-bgf7g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.526569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.526465 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.526569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.526506 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08fa115b-8e06-4399-ab65-657c78368182-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.526760 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.526581 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.526760 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.526659 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.576513 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.576483 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" event={"ID":"2c95c7f7-7920-4be6-96b4-b93b17a36c68","Type":"ContainerStarted","Data":"dd6bb5ca6508ae62b18776312085228ca948501d58fd61fe700be4d434de11c0"} Oct 30 18:40:13.576645 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.576527 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" event={"ID":"2c95c7f7-7920-4be6-96b4-b93b17a36c68","Type":"ContainerStarted","Data":"3ff50bac7c92c041e24a424ebcb42d7d4acdd13d38d1dbfe2c607827ca704bd4"} Oct 30 18:40:13.618375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.618288 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" podStartSLOduration=2.43510067 podStartE2EDuration="2.618269566s" totalImagesPullingTime="183.168896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:13.616860602 +0000 UTC m=+484.155052808" watchObservedRunningTime="2026-10-30 18:40:13.618269566 +0000 UTC m=+484.156461771" Oct 30 18:40:13.627643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.627592 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.627836 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.627665 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.627836 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.627709 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.627836 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.627806 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgf7g\" (UniqueName: \"kubernetes.io/projected/08fa115b-8e06-4399-ab65-657c78368182-kube-api-access-bgf7g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.628041 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.627894 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.628041 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.627929 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08fa115b-8e06-4399-ab65-657c78368182-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.628249 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.628209 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.628304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.628214 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.628483 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.628462 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.630325 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.630306 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08fa115b-8e06-4399-ab65-657c78368182-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.630405 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.630392 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08fa115b-8e06-4399-ab65-657c78368182-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.650579 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.650553 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgf7g\" (UniqueName: \"kubernetes.io/projected/08fa115b-8e06-4399-ab65-657c78368182-kube-api-access-bgf7g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4\" (UID: \"08fa115b-8e06-4399-ab65-657c78368182\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:13.744729 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:13.744646 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:14.035441 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:14.035409 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4"] Oct 30 18:40:14.040338 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:40:14.040308 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08fa115b_8e06_4399_ab65_657c78368182.slice/crio-c2705ce1eae0515cecaf3677753851a9932796f4e7244a925a5a2d4162937019 WatchSource:0}: Error finding container c2705ce1eae0515cecaf3677753851a9932796f4e7244a925a5a2d4162937019: Status 404 returned error can't find the container with id c2705ce1eae0515cecaf3677753851a9932796f4e7244a925a5a2d4162937019 Oct 30 18:40:14.291259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:14.291199 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:14.395410 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:40:14.395369 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:40:14.582855 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:14.582763 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" event={"ID":"08fa115b-8e06-4399-ab65-657c78368182","Type":"ContainerStarted","Data":"0dd9d174edebf747dc5ed5fbe7a2fe00ab661f1ad3c8152ec97cfd2ff27609f1"} Oct 30 18:40:14.582855 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:14.582827 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" event={"ID":"08fa115b-8e06-4399-ab65-657c78368182","Type":"ContainerStarted","Data":"c2705ce1eae0515cecaf3677753851a9932796f4e7244a925a5a2d4162937019"} Oct 30 18:40:14.626202 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:14.626145 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" podStartSLOduration=1.439635488 podStartE2EDuration="1.626130026s" totalImagesPullingTime="186.494538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:14.624753941 +0000 UTC m=+485.162946147" watchObservedRunningTime="2026-10-30 18:40:14.626130026 +0000 UTC m=+485.164322231" Oct 30 18:40:14.745367 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:14.745339 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:15.045984 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.045928 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-g98cv"] Oct 30 18:40:15.049700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.049671 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.064223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.064195 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\"" Oct 30 18:40:15.077206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.077171 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-g98cv"] Oct 30 18:40:15.243168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.243133 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.243343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.243183 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfdhg\" (UniqueName: \"kubernetes.io/projected/4d1ba668-32cf-47c9-9566-f0de15154a1c-kube-api-access-bfdhg\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.243343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.243288 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.243343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.243314 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.243506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.243354 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d1ba668-32cf-47c9-9566-f0de15154a1c-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.243506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.243475 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-home\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344012 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdhg\" (UniqueName: \"kubernetes.io/projected/4d1ba668-32cf-47c9-9566-f0de15154a1c-kube-api-access-bfdhg\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344055 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344072 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344117 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d1ba668-32cf-47c9-9566-f0de15154a1c-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344139 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-home\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344200 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344555 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344535 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-home\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344537 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.344615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.344578 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.346297 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.346278 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d1ba668-32cf-47c9-9566-f0de15154a1c-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.346560 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.346543 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d1ba668-32cf-47c9-9566-f0de15154a1c-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.367500 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.367477 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfdhg\" (UniqueName: \"kubernetes.io/projected/4d1ba668-32cf-47c9-9566-f0de15154a1c-kube-api-access-bfdhg\") pod \"echo-simulator-kserve-64b6f8f96c-g98cv\" (UID: \"4d1ba668-32cf-47c9-9566-f0de15154a1c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.663380 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.663288 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:15.958405 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:15.958320 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-g98cv"] Oct 30 18:40:15.962491 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:40:15.962460 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1ba668_32cf_47c9_9566_f0de15154a1c.slice/crio-646397c53da166373c17e034b67f88a9d12269ddadf930b87e8aaf0c189e1f5e WatchSource:0}: Error finding container 646397c53da166373c17e034b67f88a9d12269ddadf930b87e8aaf0c189e1f5e: Status 404 returned error can't find the container with id 646397c53da166373c17e034b67f88a9d12269ddadf930b87e8aaf0c189e1f5e Oct 30 18:40:16.194356 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:16.194321 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:40:16.594920 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:16.594840 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" event={"ID":"4d1ba668-32cf-47c9-9566-f0de15154a1c","Type":"ContainerStarted","Data":"646397c53da166373c17e034b67f88a9d12269ddadf930b87e8aaf0c189e1f5e"} Oct 30 18:40:18.604892 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:18.604860 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" event={"ID":"4d1ba668-32cf-47c9-9566-f0de15154a1c","Type":"ContainerStarted","Data":"c7c4ce94a954f26e138fabdae0abc0149407c8757bde3622eea6c59be313af71"} Oct 30 18:40:18.660373 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:18.660322 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" podStartSLOduration=2.058406627 podStartE2EDuration="3.66030847s" totalImagesPullingTime="1.601901843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:18.659361781 +0000 UTC m=+489.197553987" watchObservedRunningTime="2026-10-30 18:40:18.66030847 +0000 UTC m=+489.198500653" Oct 30 18:40:18.664108 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:18.664086 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:22.291722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:22.291682 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:22.303779 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:22.303753 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:22.636235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:22.636157 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xvbrz" Oct 30 18:40:23.745878 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:23.745850 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:23.758117 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:23.758094 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:24.084179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:24.084143 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:40:24.645235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:24.645208 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4" Oct 30 18:40:25.664021 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:25.663988 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:25.676282 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:25.676256 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:26.660189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:26.660156 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-g98cv" Oct 30 18:40:38.124188 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.124153 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9"] Oct 30 18:40:38.130878 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.130850 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.153294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.153255 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9"] Oct 30 18:40:38.245187 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.245157 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9bw4x\"" Oct 30 18:40:38.253557 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253526 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253697 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253564 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3578481-4db4-4032-9949-49c93f833fe8-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253697 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253597 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253697 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253686 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3578481-4db4-4032-9949-49c93f833fe8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253821 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253804 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3578481-4db4-4032-9949-49c93f833fe8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253859 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253836 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253893 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253877 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253934 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253921 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.253984 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.253947 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gvrl\" (UniqueName: \"kubernetes.io/projected/f3578481-4db4-4032-9949-49c93f833fe8-kube-api-access-8gvrl\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355027 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.354982 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355044 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3578481-4db4-4032-9949-49c93f833fe8-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355082 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355109 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3578481-4db4-4032-9949-49c93f833fe8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355191 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3578481-4db4-4032-9949-49c93f833fe8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355431 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355238 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355431 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355304 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355431 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355404 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355578 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355433 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gvrl\" (UniqueName: \"kubernetes.io/projected/f3578481-4db4-4032-9949-49c93f833fe8-kube-api-access-8gvrl\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.355637 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.355584 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.356113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.356063 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3578481-4db4-4032-9949-49c93f833fe8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.360179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.360148 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.360668 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.360620 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.361152 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.361132 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.362595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.362571 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3578481-4db4-4032-9949-49c93f833fe8-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.362805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.362784 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3578481-4db4-4032-9949-49c93f833fe8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.380475 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.380421 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3578481-4db4-4032-9949-49c93f833fe8-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.380818 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.380797 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gvrl\" (UniqueName: \"kubernetes.io/projected/f3578481-4db4-4032-9949-49c93f833fe8-kube-api-access-8gvrl\") pod \"maas-default-gateway-openshift-default-6dcf757899-q9ng9\" (UID: \"f3578481-4db4-4032-9949-49c93f833fe8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.444526 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.444489 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:38.763229 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.763131 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9"] Oct 30 18:40:38.766925 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:40:38.766900 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3578481_4db4_4032_9949_49c93f833fe8.slice/crio-a0b8c26deb7fb68e139440f7c362d1a2a92f956f20d6808805f3306bc6609f69 WatchSource:0}: Error finding container a0b8c26deb7fb68e139440f7c362d1a2a92f956f20d6808805f3306bc6609f69: Status 404 returned error can't find the container with id a0b8c26deb7fb68e139440f7c362d1a2a92f956f20d6808805f3306bc6609f69 Oct 30 18:40:38.768983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:38.768945 6160 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:40:39.704694 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:39.704660 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" event={"ID":"f3578481-4db4-4032-9949-49c93f833fe8","Type":"ContainerStarted","Data":"a0b8c26deb7fb68e139440f7c362d1a2a92f956f20d6808805f3306bc6609f69"} Oct 30 18:40:41.580518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:41.580487 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:40:41.580762 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:41.580573 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:40:41.580762 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:41.580602 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:40:41.715987 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:41.715933 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" event={"ID":"f3578481-4db4-4032-9949-49c93f833fe8","Type":"ContainerStarted","Data":"19cbd1dc4cebc42153ab09c4305ceb248291b908b9eb5f76812ffa5a472f58e3"} Oct 30 18:40:41.773923 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:41.773824 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podStartSLOduration=0.963347183 podStartE2EDuration="3.773778568s" totalImagesPullingTime="2.810431385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:41.771641739 +0000 UTC m=+512.309833946" watchObservedRunningTime="2026-10-30 18:40:41.773778568 +0000 UTC m=+512.311970774" Oct 30 18:40:42.445523 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:42.445488 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:42.446871 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:42.446845 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:42.447034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:42.446908 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:43.445486 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:43.445446 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:43.445893 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:43.445511 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:44.401044 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:40:44.401010 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:40:44.445685 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:44.445653 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:44.446056 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:44.445703 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:45.445881 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:45.445842 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:45.446333 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:45.445897 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:46.445138 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:46.445106 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:46.445311 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:46.445158 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:47.445803 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:47.445760 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:47.446329 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:47.445833 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:48.445124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:48.445086 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-q9ng9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Oct 30 18:40:48.445299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:48.445155 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" podUID="f3578481-4db4-4032-9949-49c93f833fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Oct 30 18:40:49.754526 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:49.754496 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:49.755017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:49.754879 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:40:49.755688 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:40:49.755671 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-q9ng9" Oct 30 18:41:11.199834 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.199796 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl"] Oct 30 18:41:11.205830 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.205811 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.224435 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.224411 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b77c9bba-serving-cert\"" Oct 30 18:41:11.227281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.227190 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl"] Oct 30 18:41:11.362931 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.362896 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.362931 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.362937 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84fc6\" (UniqueName: \"kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.363168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.362983 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.363234 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.363169 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.364208 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.364183 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf"] Oct 30 18:41:11.368069 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.368051 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.383153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.383130 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b77c9bba\"" Oct 30 18:41:11.383153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.383149 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b77c9bba-dockercfg-bwvk2\"" Oct 30 18:41:11.388543 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.388522 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf"] Oct 30 18:41:11.464597 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.464508 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.464597 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.464584 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj787\" (UniqueName: \"kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787\") pod \"payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.464827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.464651 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.464827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.464693 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.464827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.464789 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.464827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.464823 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84fc6\" (UniqueName: \"kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.466382 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.466351 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.469477 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.469452 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.469622 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.469531 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.471890 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.471867 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc"] Oct 30 18:41:11.476190 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.476171 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.492122 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.492104 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84fc6\" (UniqueName: \"kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6\") pod \"maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.497586 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.497561 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc"] Oct 30 18:41:11.520341 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.520324 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:11.565551 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.565517 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljwft\" (UniqueName: \"kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft\") pod \"payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.565715 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.565569 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj787\" (UniqueName: \"kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787\") pod \"payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.565715 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.565632 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.565715 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.565709 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.566737 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.566714 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.590519 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.590493 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj787\" (UniqueName: \"kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787\") pod \"payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.666643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.666610 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljwft\" (UniqueName: \"kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft\") pod \"payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.666801 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.666670 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.667395 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.667366 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.681530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.681503 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:11.690194 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.690166 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljwft\" (UniqueName: \"kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft\") pod \"payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.797816 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.797784 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:11.819974 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.819927 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl"] Oct 30 18:41:11.823921 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:11.823898 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d4a624_4470_408a_9198_cfa19d88b744.slice/crio-8c895e5a28e6d88fbfe3147edbf29a3fbb26b97676cfbe357d48784d9a9e0c71 WatchSource:0}: Error finding container 8c895e5a28e6d88fbfe3147edbf29a3fbb26b97676cfbe357d48784d9a9e0c71: Status 404 returned error can't find the container with id 8c895e5a28e6d88fbfe3147edbf29a3fbb26b97676cfbe357d48784d9a9e0c71 Oct 30 18:41:11.853988 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:11.852405 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" event={"ID":"13d4a624-4470-408a-9198-cfa19d88b744","Type":"ContainerStarted","Data":"8c895e5a28e6d88fbfe3147edbf29a3fbb26b97676cfbe357d48784d9a9e0c71"} Oct 30 18:41:12.012735 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.012647 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf"] Oct 30 18:41:12.016038 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:12.016008 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd54e50c_3a59_4169_82cf_cff4ff3a7d07.slice/crio-221ede66d6fb8eddb384eccceb637f1b325372eeaa61a45d5d98a03d9e6b28cd WatchSource:0}: Error finding container 221ede66d6fb8eddb384eccceb637f1b325372eeaa61a45d5d98a03d9e6b28cd: Status 404 returned error can't find the container with id 221ede66d6fb8eddb384eccceb637f1b325372eeaa61a45d5d98a03d9e6b28cd Oct 30 18:41:12.096206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.096179 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc"] Oct 30 18:41:12.100932 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:12.100905 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e685408_7570_4784_9e34_0067f64326d0.slice/crio-28638df2993df155534718ec5b669907b09ad07cf5069bab2e8bef7c4a0c9efb WatchSource:0}: Error finding container 28638df2993df155534718ec5b669907b09ad07cf5069bab2e8bef7c4a0c9efb: Status 404 returned error can't find the container with id 28638df2993df155534718ec5b669907b09ad07cf5069bab2e8bef7c4a0c9efb Oct 30 18:41:12.859986 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.859929 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" event={"ID":"13d4a624-4470-408a-9198-cfa19d88b744","Type":"ContainerStarted","Data":"37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2"} Oct 30 18:41:12.860462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.860026 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:12.861628 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.861598 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" event={"ID":"5e685408-7570-4784-9e34-0067f64326d0","Type":"ContainerStarted","Data":"5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc"} Oct 30 18:41:12.861757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.861646 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" event={"ID":"5e685408-7570-4784-9e34-0067f64326d0","Type":"ContainerStarted","Data":"28638df2993df155534718ec5b669907b09ad07cf5069bab2e8bef7c4a0c9efb"} Oct 30 18:41:12.861757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.861674 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:12.863102 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.863074 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" event={"ID":"dd54e50c-3a59-4169-82cf-cff4ff3a7d07","Type":"ContainerStarted","Data":"4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b"} Oct 30 18:41:12.863241 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.863123 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" event={"ID":"dd54e50c-3a59-4169-82cf-cff4ff3a7d07","Type":"ContainerStarted","Data":"221ede66d6fb8eddb384eccceb637f1b325372eeaa61a45d5d98a03d9e6b28cd"} Oct 30 18:41:12.863241 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.863213 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:12.910556 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.910494 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" podStartSLOduration=1.468197108 podStartE2EDuration="1.910478128s" totalImagesPullingTime="442.28102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:12.907861179 +0000 UTC m=+543.446053389" watchObservedRunningTime="2026-10-30 18:41:12.910478128 +0000 UTC m=+543.448670332" Oct 30 18:41:12.955651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:12.955601 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" podStartSLOduration=1.9555872779999999 podStartE2EDuration="1.955587278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:12.954228044 +0000 UTC m=+543.492420249" watchObservedRunningTime="2026-10-30 18:41:12.955587278 +0000 UTC m=+543.493779500" Oct 30 18:41:13.006909 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.006857 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" podStartSLOduration=2.006841268 podStartE2EDuration="2.006841268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:13.005283453 +0000 UTC m=+543.543475658" watchObservedRunningTime="2026-10-30 18:41:13.006841268 +0000 UTC m=+543.545033474" Oct 30 18:41:13.083274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.083240 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt"] Oct 30 18:41:13.086948 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.086923 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.109694 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.109650 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f7d8050f\"" Oct 30 18:41:13.110036 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.109980 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f7d8050f-dockercfg-mbk9l\"" Oct 30 18:41:13.125504 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.125469 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt"] Oct 30 18:41:13.179608 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.179568 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stk68\" (UniqueName: \"kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68\") pod \"payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.179926 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.179900 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.202339 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.202290 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs"] Oct 30 18:41:13.206057 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.206036 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.280380 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.280347 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume\") pod \"payload-processing-e2e-ait-f7d8050f-9477df678-26mhs\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.280380 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.280384 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stk68\" (UniqueName: \"kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68\") pod \"payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.280641 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.280494 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.280641 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.280547 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdcrn\" (UniqueName: \"kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn\") pod \"payload-processing-e2e-ait-f7d8050f-9477df678-26mhs\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.281261 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.281242 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.310542 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.310508 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stk68\" (UniqueName: \"kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68\") pod \"payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.316246 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.316216 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs"] Oct 30 18:41:13.381619 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.381488 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdcrn\" (UniqueName: \"kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn\") pod \"payload-processing-e2e-ait-f7d8050f-9477df678-26mhs\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.381619 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.381574 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume\") pod \"payload-processing-e2e-ait-f7d8050f-9477df678-26mhs\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.382275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.382254 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume\") pod \"payload-processing-e2e-ait-f7d8050f-9477df678-26mhs\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.400046 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.400020 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.408631 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.408605 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdcrn\" (UniqueName: \"kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn\") pod \"payload-processing-e2e-ait-f7d8050f-9477df678-26mhs\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.519097 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.519064 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:13.699725 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.699648 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt"] Oct 30 18:41:13.703748 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:13.703724 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ad376f_2b5f_4907_8275_395d302442f8.slice/crio-7db1f59a5436da24cd88b87795a643fb910591c5f0f76be94d117befb3eb2182 WatchSource:0}: Error finding container 7db1f59a5436da24cd88b87795a643fb910591c5f0f76be94d117befb3eb2182: Status 404 returned error can't find the container with id 7db1f59a5436da24cd88b87795a643fb910591c5f0f76be94d117befb3eb2182 Oct 30 18:41:13.815415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.815382 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs"] Oct 30 18:41:13.822632 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:13.822601 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57cb811d_cdf5_47aa_abf0_89b67b6403fd.slice/crio-454af26f0fd578d4666f65da6e860a0c31bc3f1eca8349d5ce9349df6c5314d7 WatchSource:0}: Error finding container 454af26f0fd578d4666f65da6e860a0c31bc3f1eca8349d5ce9349df6c5314d7: Status 404 returned error can't find the container with id 454af26f0fd578d4666f65da6e860a0c31bc3f1eca8349d5ce9349df6c5314d7 Oct 30 18:41:13.868795 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.868766 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" event={"ID":"57cb811d-cdf5-47aa-abf0-89b67b6403fd","Type":"ContainerStarted","Data":"454af26f0fd578d4666f65da6e860a0c31bc3f1eca8349d5ce9349df6c5314d7"} Oct 30 18:41:13.872799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.870616 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" event={"ID":"f8ad376f-2b5f-4907-8275-395d302442f8","Type":"ContainerStarted","Data":"13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f"} Oct 30 18:41:13.872799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.870669 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" event={"ID":"f8ad376f-2b5f-4907-8275-395d302442f8","Type":"ContainerStarted","Data":"7db1f59a5436da24cd88b87795a643fb910591c5f0f76be94d117befb3eb2182"} Oct 30 18:41:13.872799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.871103 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:13.914189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:13.914094 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" podStartSLOduration=0.914080033 podStartE2EDuration="914.080033ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:13.913816796 +0000 UTC m=+544.452009000" watchObservedRunningTime="2026-10-30 18:41:13.914080033 +0000 UTC m=+544.452272238" Oct 30 18:41:14.159504 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.159468 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk"] Oct 30 18:41:14.163385 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.163365 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.187720 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.187688 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk"] Oct 30 18:41:14.190001 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.189974 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-48afdd9e\"" Oct 30 18:41:14.190996 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.190580 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-48afdd9e-dockercfg-sjjmv\"" Oct 30 18:41:14.191439 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.191243 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.191439 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.191312 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzs6s\" (UniqueName: \"kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s\") pod \"payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.216806 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.216770 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr"] Oct 30 18:41:14.222699 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.222672 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.239333 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.239305 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr"] Oct 30 18:41:14.292390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.292353 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzs6s\" (UniqueName: \"kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s\") pod \"payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.292569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.292515 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.292569 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.292550 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume\") pod \"payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.292662 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.292576 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjfn7\" (UniqueName: \"kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7\") pod \"payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.293530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.293503 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.316092 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.316061 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzs6s\" (UniqueName: \"kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s\") pod \"payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.393491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.393447 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume\") pod \"payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.393491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.393481 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjfn7\" (UniqueName: \"kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7\") pod \"payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.394224 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.394199 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume\") pod \"payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.395043 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:14.395017 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:41:14.418713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.418657 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjfn7\" (UniqueName: \"kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7\") pod \"payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.476713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.476678 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.537349 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.537317 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:14.548862 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.548831 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq"] Oct 30 18:41:14.554123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.554101 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.571925 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.571887 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a33e5151-openshift-default-dockercfg-857xw\"" Oct 30 18:41:14.581014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.579689 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq"] Oct 30 18:41:14.696155 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696080 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696155 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696117 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696293 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696343 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4lmq\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696465 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696442 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696523 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696509 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696560 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696534 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696560 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696552 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.696633 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.696594 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.771091 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.771055 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk"] Oct 30 18:41:14.775941 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:14.775906 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae98a480_a3c6_4131_85b4_cbb77e1dc2d7.slice/crio-eb178bac2d21b73787c5a8e40d7fd0c7773346b0047728269e3f8d53a6db0a19 WatchSource:0}: Error finding container eb178bac2d21b73787c5a8e40d7fd0c7773346b0047728269e3f8d53a6db0a19: Status 404 returned error can't find the container with id eb178bac2d21b73787c5a8e40d7fd0c7773346b0047728269e3f8d53a6db0a19 Oct 30 18:41:14.797438 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797402 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797602 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797457 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797602 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797494 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797602 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797525 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797602 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797589 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797622 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797691 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797727 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4lmq\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.797866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797772 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.798102 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.797978 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.798213 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.798188 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.798276 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.798241 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.799609 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.799583 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.799929 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.799902 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.800866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.800779 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.801314 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.801285 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.821290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.821265 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.821441 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.821424 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4lmq\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq\") pod \"e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.856128 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.856100 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr"] Oct 30 18:41:14.859623 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:14.859591 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eac19ae_a9a2_4685_b724_52512b4c5de0.slice/crio-e091b87599b61adec5a8c521dd694c503e24a5ce4b4efa052a45e253b83db88b WatchSource:0}: Error finding container e091b87599b61adec5a8c521dd694c503e24a5ce4b4efa052a45e253b83db88b: Status 404 returned error can't find the container with id e091b87599b61adec5a8c521dd694c503e24a5ce4b4efa052a45e253b83db88b Oct 30 18:41:14.877603 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.877561 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8"] Oct 30 18:41:14.882847 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882826 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" event={"ID":"57cb811d-cdf5-47aa-abf0-89b67b6403fd","Type":"ContainerStarted","Data":"f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8"} Oct 30 18:41:14.882946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882856 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" event={"ID":"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7","Type":"ContainerStarted","Data":"baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05"} Oct 30 18:41:14.882946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882877 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:14.882946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882899 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:14.882946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882918 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" event={"ID":"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7","Type":"ContainerStarted","Data":"eb178bac2d21b73787c5a8e40d7fd0c7773346b0047728269e3f8d53a6db0a19"} Oct 30 18:41:14.882946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882939 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" event={"ID":"4eac19ae-a9a2-4685-b724-52512b4c5de0","Type":"ContainerStarted","Data":"e091b87599b61adec5a8c521dd694c503e24a5ce4b4efa052a45e253b83db88b"} Oct 30 18:41:14.883122 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.882966 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:14.898046 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.898026 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:14.898369 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.898347 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p76x\" (UniqueName: \"kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:14.898477 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.898456 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:14.898596 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.898579 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:14.898646 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.898611 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:14.902460 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.902409 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f7d8050f-serving-cert\"" Oct 30 18:41:14.904657 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.904635 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8"] Oct 30 18:41:14.919052 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.919005 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" podStartSLOduration=1.9189883760000002 podStartE2EDuration="1.918988376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:14.918728313 +0000 UTC m=+545.456920517" watchObservedRunningTime="2026-10-30 18:41:14.918988376 +0000 UTC m=+545.457180581" Oct 30 18:41:14.961994 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.961838 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" podStartSLOduration=0.961817471 podStartE2EDuration="961.817471ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:14.959370759 +0000 UTC m=+545.497562966" watchObservedRunningTime="2026-10-30 18:41:14.961817471 +0000 UTC m=+545.500009680" Oct 30 18:41:14.999918 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.999848 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p76x\" (UniqueName: \"kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.000065 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:14.999938 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.000065 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.000050 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.000185 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.000078 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.000933 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.000886 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.003062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.003032 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.003722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.003701 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.024709 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.024689 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p76x\" (UniqueName: \"kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x\") pod \"maas-api-e2e-ait-f7d8050f-84458575b-w8kk8\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.211275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.211235 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:15.224353 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.224334 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq"] Oct 30 18:41:15.228919 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:15.228890 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99097643_a6bb_46f4_8804_9b10d8317a28.slice/crio-0665bf358cc1735c3b259e82703fb44e087e766a353bcacadc2f357e1b89b27c WatchSource:0}: Error finding container 0665bf358cc1735c3b259e82703fb44e087e766a353bcacadc2f357e1b89b27c: Status 404 returned error can't find the container with id 0665bf358cc1735c3b259e82703fb44e087e766a353bcacadc2f357e1b89b27c Oct 30 18:41:15.231390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.231355 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:15.231466 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.231441 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:15.231505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.231489 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:15.515153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.515057 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8"] Oct 30 18:41:15.519638 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:15.519606 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d85ef0a_e693_4779_9ca7_57ff89a6ec96.slice/crio-ab7183d8c2d60658f3e62397accdc3821169e6438628c21a749c75c5d0eb3ec0 WatchSource:0}: Error finding container ab7183d8c2d60658f3e62397accdc3821169e6438628c21a749c75c5d0eb3ec0: Status 404 returned error can't find the container with id ab7183d8c2d60658f3e62397accdc3821169e6438628c21a749c75c5d0eb3ec0 Oct 30 18:41:15.888314 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.888276 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" event={"ID":"99097643-a6bb-46f4-8804-9b10d8317a28","Type":"ContainerStarted","Data":"5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58"} Oct 30 18:41:15.888651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.888345 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" event={"ID":"99097643-a6bb-46f4-8804-9b10d8317a28","Type":"ContainerStarted","Data":"0665bf358cc1735c3b259e82703fb44e087e766a353bcacadc2f357e1b89b27c"} Oct 30 18:41:15.897536 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.897507 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" event={"ID":"4eac19ae-a9a2-4685-b724-52512b4c5de0","Type":"ContainerStarted","Data":"9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831"} Oct 30 18:41:15.897662 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.897613 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:15.898355 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.898339 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:15.899391 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.899369 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" event={"ID":"6d85ef0a-e693-4779-9ca7-57ff89a6ec96","Type":"ContainerStarted","Data":"ab7183d8c2d60658f3e62397accdc3821169e6438628c21a749c75c5d0eb3ec0"} Oct 30 18:41:15.936285 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.936232 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podStartSLOduration=1.936214288 podStartE2EDuration="1.936214288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:15.934082822 +0000 UTC m=+546.472275027" watchObservedRunningTime="2026-10-30 18:41:15.936214288 +0000 UTC m=+546.474406495" Oct 30 18:41:15.975779 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.975710 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" podStartSLOduration=1.975684982 podStartE2EDuration="1.975684982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:15.975055489 +0000 UTC m=+546.513247711" watchObservedRunningTime="2026-10-30 18:41:15.975684982 +0000 UTC m=+546.513877273" Oct 30 18:41:15.983381 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:15.983351 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:41:16.905017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:16.904977 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:16.906472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:16.906445 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" event={"ID":"6d85ef0a-e693-4779-9ca7-57ff89a6ec96","Type":"ContainerStarted","Data":"044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92"} Oct 30 18:41:16.906880 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:16.906853 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:16.906996 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:16.906908 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:16.907520 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:16.907500 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:17.069122 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:17.068885 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" podStartSLOduration=2.738479971 podStartE2EDuration="3.068866468s" totalImagesPullingTime="330.386497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:17.066905718 +0000 UTC m=+547.605097923" watchObservedRunningTime="2026-10-30 18:41:17.068866468 +0000 UTC m=+547.607058673" Oct 30 18:41:18.477854 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.477821 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj"] Oct 30 18:41:18.484446 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.484425 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.503236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.503109 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj"] Oct 30 18:41:18.512910 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.512885 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-48afdd9e-serving-cert\"" Oct 30 18:41:18.543428 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.543398 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.543587 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.543557 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.543689 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.543636 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.543689 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.543672 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dmh7\" (UniqueName: \"kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.645539 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.645507 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.645726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.645549 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.645726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.645593 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dmh7\" (UniqueName: \"kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.645726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.645662 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.645726 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:18.645685 6160 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-serving-cert: secret "maas-api-e2e-ait-48afdd9e-serving-cert" not found Oct 30 18:41:18.645938 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:18.645756 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls podName:4e78b814-bb78-43e8-9c91-40ab995f77b8 nodeName:}" failed. No retries permitted until 2026-10-30 18:41:19.145736732 +0000 UTC m=+549.683928934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls") pod "maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" (UID: "4e78b814-bb78-43e8-9c91-40ab995f77b8") : secret "maas-api-e2e-ait-48afdd9e-serving-cert" not found Oct 30 18:41:18.646402 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.646380 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.648011 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.647988 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.682075 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.682040 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dmh7\" (UniqueName: \"kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:18.876066 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:18.876034 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:19.151661 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:19.151587 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:19.154109 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:19.154084 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\") pod \"maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:19.398427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:19.397889 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:19.730682 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:19.730607 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj"] Oct 30 18:41:19.734389 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:19.734360 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e78b814_bb78_43e8_9c91_40ab995f77b8.slice/crio-4452e9b3b3faacafd15fd180212c6fed083b567d33a7fd25ae8e293acb5218f3 WatchSource:0}: Error finding container 4452e9b3b3faacafd15fd180212c6fed083b567d33a7fd25ae8e293acb5218f3: Status 404 returned error can't find the container with id 4452e9b3b3faacafd15fd180212c6fed083b567d33a7fd25ae8e293acb5218f3 Oct 30 18:41:19.923185 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:19.923151 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" event={"ID":"4e78b814-bb78-43e8-9c91-40ab995f77b8","Type":"ContainerStarted","Data":"4452e9b3b3faacafd15fd180212c6fed083b567d33a7fd25ae8e293acb5218f3"} Oct 30 18:41:20.533672 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.533638 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb"] Oct 30 18:41:20.537307 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.537283 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.560683 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.560651 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb"] Oct 30 18:41:20.566092 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.566064 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.566235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.566144 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.566235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.566191 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.566367 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.566327 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk7s4\" (UniqueName: \"kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.611208 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.611169 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ed302a08-serving-cert\"" Oct 30 18:41:20.667706 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.667669 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.667898 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.667814 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.667898 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.667859 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.667898 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.667889 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk7s4\" (UniqueName: \"kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.668584 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.668549 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.670305 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.670280 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.670594 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.670574 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.691180 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.691156 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk7s4\" (UniqueName: \"kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4\") pod \"maas-api-e2e-ait-ed302a08-878cf8b89-96vfb\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.850024 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.849884 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:20.930361 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.930317 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" event={"ID":"4e78b814-bb78-43e8-9c91-40ab995f77b8","Type":"ContainerStarted","Data":"1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3"} Oct 30 18:41:20.930551 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.930499 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:20.973545 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:20.973489 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" podStartSLOduration=2.686385452 podStartE2EDuration="2.973475151s" totalImagesPullingTime="287.089699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:20.971540811 +0000 UTC m=+551.509733015" watchObservedRunningTime="2026-10-30 18:41:20.973475151 +0000 UTC m=+551.511667364" Oct 30 18:41:21.155168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:21.155076 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb"] Oct 30 18:41:21.937017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:21.936918 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" event={"ID":"bd22b256-c1c6-4040-8f56-b67e405a0782","Type":"ContainerStarted","Data":"c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5"} Oct 30 18:41:21.937017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:21.937006 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" event={"ID":"bd22b256-c1c6-4040-8f56-b67e405a0782","Type":"ContainerStarted","Data":"7b6d3679523002d2ef45bb833645d7306c26da51d0091412df5a8605d0ae846c"} Oct 30 18:41:21.937429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:21.937134 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:21.982851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:21.982803 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" podStartSLOduration=1.5974129609999999 podStartE2EDuration="1.982788381s" totalImagesPullingTime="385.37542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:21.98192726 +0000 UTC m=+552.520119466" watchObservedRunningTime="2026-10-30 18:41:21.982788381 +0000 UTC m=+552.520980586" Oct 30 18:41:22.918593 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:22.918561 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:23.874305 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:23.874273 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:23.874807 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:23.874357 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:25.191210 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.191174 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:41:25.871182 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.871149 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw"] Oct 30 18:41:25.875189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.875166 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:25.892887 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.892864 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a33e5151-serving-cert\"" Oct 30 18:41:25.901028 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.901002 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw"] Oct 30 18:41:25.908276 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.908224 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:25.908636 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.908619 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:25.909328 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.909310 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:25.922624 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.922602 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:25.922703 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.922684 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4kkb\" (UniqueName: \"kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:25.922764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.922717 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:25.922827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:25.922812 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.023630 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.023587 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4kkb\" (UniqueName: \"kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.023824 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.023663 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.023908 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.023830 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.024034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.024013 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.025405 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.025375 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.026743 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.026717 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.027101 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.027076 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.049066 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.049033 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4kkb\" (UniqueName: \"kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb\") pod \"maas-api-e2e-derive-a33e5151-85cfd7649-94mqw\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.191119 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.191023 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:26.503271 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.503186 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw"] Oct 30 18:41:26.506792 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:26.506761 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda70d7f5_08f8_48ec_ad43_3926eb875ef4.slice/crio-fbbb0a9cc354a144a96c2ba1a22a02a50257e728448dd0df23d914b225a37fb4 WatchSource:0}: Error finding container fbbb0a9cc354a144a96c2ba1a22a02a50257e728448dd0df23d914b225a37fb4: Status 404 returned error can't find the container with id fbbb0a9cc354a144a96c2ba1a22a02a50257e728448dd0df23d914b225a37fb4 Oct 30 18:41:26.909894 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.909866 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:26.942640 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.942613 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:26.977547 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.977510 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" event={"ID":"da70d7f5-08f8-48ec-ad43-3926eb875ef4","Type":"ContainerStarted","Data":"fbbb0a9cc354a144a96c2ba1a22a02a50257e728448dd0df23d914b225a37fb4"} Oct 30 18:41:26.977788 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:26.977766 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:27.040840 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:27.040781 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" podStartSLOduration=1.663703442 podStartE2EDuration="2.040746512s" totalImagesPullingTime="377.04307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:27.039891439 +0000 UTC m=+557.578083643" watchObservedRunningTime="2026-10-30 18:41:27.040746512 +0000 UTC m=+557.578938717" Oct 30 18:41:27.947666 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:27.947638 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:27.984123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:27.984087 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" event={"ID":"da70d7f5-08f8-48ec-ad43-3926eb875ef4","Type":"ContainerStarted","Data":"3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410"} Oct 30 18:41:33.601608 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:33.601564 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:41:33.995494 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:33.995420 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:40.046610 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.046582 6160 generic.go:356] "Generic (PLEG): container finished" podID="99097643-a6bb-46f4-8804-9b10d8317a28" containerID="5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58" exitCode=0 Oct 30 18:41:40.046939 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.046653 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" event={"ID":"99097643-a6bb-46f4-8804-9b10d8317a28","Type":"ContainerDied","Data":"5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58"} Oct 30 18:41:40.047064 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.047049 6160 scope.go:122] "RemoveContainer" containerID="5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58" Oct 30 18:41:40.048281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.048254 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:40.048357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.048321 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:40.048357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.048351 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:40.443193 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.443161 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc"] Oct 30 18:41:40.474862 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:40.474832 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf"] Oct 30 18:41:41.088126 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:41.088086 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" event={"ID":"99097643-a6bb-46f4-8804-9b10d8317a28","Type":"ContainerStarted","Data":"8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504"} Oct 30 18:41:41.778538 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:41.778494 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs"] Oct 30 18:41:41.803491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:41.803451 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt"] Oct 30 18:41:41.898728 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:41.898696 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:41.899138 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:41.899110 6160 patch_prober.go:29] interesting pod/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Oct 30 18:41:41.899267 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:41.899163 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Oct 30 18:41:42.898755 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:42.898718 6160 patch_prober.go:29] interesting pod/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Oct 30 18:41:42.899160 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:42.898784 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Oct 30 18:41:43.899388 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:43.899353 6160 patch_prober.go:29] interesting pod/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Oct 30 18:41:43.899755 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:43.899397 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Oct 30 18:41:44.403302 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:44.403266 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:41:44.905117 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:44.905081 6160 patch_prober.go:29] interesting pod/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Oct 30 18:41:44.905527 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:44.905159 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Oct 30 18:41:45.406349 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.406314 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8"] Oct 30 18:41:45.406626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.406598 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" podUID="6d85ef0a-e693-4779-9ca7-57ff89a6ec96" containerName="maas-api" containerID="cri-o://044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92" gracePeriod=30 Oct 30 18:41:45.444442 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.444384 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" podUID="5e685408-7570-4784-9e34-0067f64326d0" containerName="payload-processing" containerID="cri-o://5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc" gracePeriod=25 Oct 30 18:41:45.477084 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.477037 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" podUID="dd54e50c-3a59-4169-82cf-cff4ff3a7d07" containerName="payload-pre-processing" containerID="cri-o://4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b" gracePeriod=25 Oct 30 18:41:45.797153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.796893 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:45.896010 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.895656 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:45.899332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.898847 6160 patch_prober.go:29] interesting pod/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Oct 30 18:41:45.899332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.898918 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Oct 30 18:41:45.912719 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.912661 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:45.935897 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.935861 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs\") pod \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " Oct 30 18:41:45.936062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.935930 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls\") pod \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " Oct 30 18:41:45.936062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.935990 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x\" (UniqueName: \"kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x\") pod \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " Oct 30 18:41:45.936178 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.936113 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca\") pod \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\" (UID: \"6d85ef0a-e693-4779-9ca7-57ff89a6ec96\") " Oct 30 18:41:45.936178 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.936146 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft\" (UniqueName: \"kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft\") pod \"5e685408-7570-4784-9e34-0067f64326d0\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " Oct 30 18:41:45.936277 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.936230 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume\") pod \"5e685408-7570-4784-9e34-0067f64326d0\" (UID: \"5e685408-7570-4784-9e34-0067f64326d0\") " Oct 30 18:41:45.937015 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.936918 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume" pod "5e685408-7570-4784-9e34-0067f64326d0" (UID: "5e685408-7570-4784-9e34-0067f64326d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:45.939270 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.939240 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls" pod "6d85ef0a-e693-4779-9ca7-57ff89a6ec96" (UID: "6d85ef0a-e693-4779-9ca7-57ff89a6ec96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:45.940261 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.940239 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs" pod "6d85ef0a-e693-4779-9ca7-57ff89a6ec96" (UID: "6d85ef0a-e693-4779-9ca7-57ff89a6ec96"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:45.940515 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.940470 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca" pod "6d85ef0a-e693-4779-9ca7-57ff89a6ec96" (UID: "6d85ef0a-e693-4779-9ca7-57ff89a6ec96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:45.940515 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.940513 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x" pod "6d85ef0a-e693-4779-9ca7-57ff89a6ec96" (UID: "6d85ef0a-e693-4779-9ca7-57ff89a6ec96"). InnerVolumeSpecName "kube-api-access-7p76x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:45.942182 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:45.942152 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft" pod "5e685408-7570-4784-9e34-0067f64326d0" (UID: "5e685408-7570-4784-9e34-0067f64326d0"). InnerVolumeSpecName "kube-api-access-ljwft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:46.037301 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037270 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787\" (UniqueName: \"kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787\") pod \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " Oct 30 18:41:46.037474 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037379 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume\") pod \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\" (UID: \"dd54e50c-3a59-4169-82cf-cff4ff3a7d07\") " Oct 30 18:41:46.037606 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037591 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.037643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037609 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljwft\" (UniqueName: \"kubernetes.io/projected/5e685408-7570-4784-9e34-0067f64326d0-kube-api-access-ljwft\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.037643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037619 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e685408-7570-4784-9e34-0067f64326d0-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.037643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037627 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.037643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037637 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.037768 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037645 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p76x\" (UniqueName: \"kubernetes.io/projected/6d85ef0a-e693-4779-9ca7-57ff89a6ec96-kube-api-access-7p76x\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.037768 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.037714 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume" pod "dd54e50c-3a59-4169-82cf-cff4ff3a7d07" (UID: "dd54e50c-3a59-4169-82cf-cff4ff3a7d07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:46.039389 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.039367 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787" pod "dd54e50c-3a59-4169-82cf-cff4ff3a7d07" (UID: "dd54e50c-3a59-4169-82cf-cff4ff3a7d07"). InnerVolumeSpecName "kube-api-access-dj787". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:46.116794 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.116755 6160 generic.go:356] "Generic (PLEG): container finished" podID="5e685408-7570-4784-9e34-0067f64326d0" containerID="5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc" exitCode=0 Oct 30 18:41:46.117009 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.116805 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" Oct 30 18:41:46.117009 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.116836 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" event={"ID":"5e685408-7570-4784-9e34-0067f64326d0","Type":"ContainerDied","Data":"5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc"} Oct 30 18:41:46.117009 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.116897 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc" event={"ID":"5e685408-7570-4784-9e34-0067f64326d0","Type":"ContainerDied","Data":"28638df2993df155534718ec5b669907b09ad07cf5069bab2e8bef7c4a0c9efb"} Oct 30 18:41:46.117009 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.116927 6160 scope.go:122] "RemoveContainer" containerID="5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc" Oct 30 18:41:46.118355 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.118333 6160 generic.go:356] "Generic (PLEG): container finished" podID="dd54e50c-3a59-4169-82cf-cff4ff3a7d07" containerID="4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b" exitCode=0 Oct 30 18:41:46.118433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.118400 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" Oct 30 18:41:46.118433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.118418 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" event={"ID":"dd54e50c-3a59-4169-82cf-cff4ff3a7d07","Type":"ContainerDied","Data":"4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b"} Oct 30 18:41:46.118529 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.118472 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf" event={"ID":"dd54e50c-3a59-4169-82cf-cff4ff3a7d07","Type":"ContainerDied","Data":"221ede66d6fb8eddb384eccceb637f1b325372eeaa61a45d5d98a03d9e6b28cd"} Oct 30 18:41:46.120102 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.120069 6160 generic.go:356] "Generic (PLEG): container finished" podID="6d85ef0a-e693-4779-9ca7-57ff89a6ec96" containerID="044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92" exitCode=0 Oct 30 18:41:46.120194 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.120126 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" Oct 30 18:41:46.120254 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.120121 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" event={"ID":"6d85ef0a-e693-4779-9ca7-57ff89a6ec96","Type":"ContainerDied","Data":"044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92"} Oct 30 18:41:46.120303 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.120256 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8" event={"ID":"6d85ef0a-e693-4779-9ca7-57ff89a6ec96","Type":"ContainerDied","Data":"ab7183d8c2d60658f3e62397accdc3821169e6438628c21a749c75c5d0eb3ec0"} Oct 30 18:41:46.127590 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.127567 6160 scope.go:122] "RemoveContainer" containerID="5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc" Oct 30 18:41:46.127893 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:46.127870 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc\": container with ID starting with 5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc not found: ID does not exist" containerID="5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc" Oct 30 18:41:46.127968 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.127904 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc"} err="failed to get container status \"5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc\": rpc error: code = NotFound desc = could not find container \"5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc\": container with ID starting with 5369c082f2493d1a8a616464a21bde09d0329e84b0722d3e2ba07d7f9d722dfc not found: ID does not exist" Oct 30 18:41:46.127968 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.127927 6160 scope.go:122] "RemoveContainer" containerID="4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b" Oct 30 18:41:46.138146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.138124 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj787\" (UniqueName: \"kubernetes.io/projected/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-kube-api-access-dj787\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.138146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.138146 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd54e50c-3a59-4169-82cf-cff4ff3a7d07-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:46.139299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.139280 6160 scope.go:122] "RemoveContainer" containerID="4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b" Oct 30 18:41:46.139530 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:46.139510 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b\": container with ID starting with 4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b not found: ID does not exist" containerID="4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b" Oct 30 18:41:46.139603 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.139534 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b"} err="failed to get container status \"4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b\": rpc error: code = NotFound desc = could not find container \"4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b\": container with ID starting with 4b69576ebadd4b0cc806696a98dbe80c9d5ab64617481c62d07041bf76f8227b not found: ID does not exist" Oct 30 18:41:46.139603 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.139550 6160 scope.go:122] "RemoveContainer" containerID="044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92" Oct 30 18:41:46.147785 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.147766 6160 scope.go:122] "RemoveContainer" containerID="044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92" Oct 30 18:41:46.148033 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:46.148009 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92\": container with ID starting with 044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92 not found: ID does not exist" containerID="044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92" Oct 30 18:41:46.148117 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.148038 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92"} err="failed to get container status \"044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92\": rpc error: code = NotFound desc = could not find container \"044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92\": container with ID starting with 044eb7f46d2ea7ab3d53b5ce9dcba51e943d09ba26d9da5845444078a8481c92 not found: ID does not exist" Oct 30 18:41:46.183585 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.183507 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc"] Oct 30 18:41:46.185738 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.185714 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b77c9bba-b86bff66-9psmc"] Oct 30 18:41:46.238965 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.238914 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8"] Oct 30 18:41:46.245258 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.245227 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7d8050f-84458575b-w8kk8"] Oct 30 18:41:46.290988 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.290924 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf"] Oct 30 18:41:46.294161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.294132 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b77c9bba-7877cfb4f5-q7tvf"] Oct 30 18:41:46.590835 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.590799 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr"] Oct 30 18:41:46.623487 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.623450 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk"] Oct 30 18:41:46.779878 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.779835 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" podUID="57cb811d-cdf5-47aa-abf0-89b67b6403fd" containerName="payload-processing" containerID="cri-o://f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8" gracePeriod=25 Oct 30 18:41:46.804448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.804409 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" podUID="f8ad376f-2b5f-4907-8275-395d302442f8" containerName="payload-pre-processing" containerID="cri-o://13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f" gracePeriod=25 Oct 30 18:41:46.903288 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.903257 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:46.903719 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.903698 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:46.904417 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:46.904395 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:41:47.079981 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.079929 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:47.092106 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.092045 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:47.131469 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.131434 6160 generic.go:356] "Generic (PLEG): container finished" podID="57cb811d-cdf5-47aa-abf0-89b67b6403fd" containerID="f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8" exitCode=0 Oct 30 18:41:47.131651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.131482 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" Oct 30 18:41:47.131651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.131523 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" event={"ID":"57cb811d-cdf5-47aa-abf0-89b67b6403fd","Type":"ContainerDied","Data":"f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8"} Oct 30 18:41:47.131651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.131586 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs" event={"ID":"57cb811d-cdf5-47aa-abf0-89b67b6403fd","Type":"ContainerDied","Data":"454af26f0fd578d4666f65da6e860a0c31bc3f1eca8349d5ce9349df6c5314d7"} Oct 30 18:41:47.131651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.131619 6160 scope.go:122] "RemoveContainer" containerID="f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8" Oct 30 18:41:47.132916 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.132891 6160 generic.go:356] "Generic (PLEG): container finished" podID="f8ad376f-2b5f-4907-8275-395d302442f8" containerID="13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f" exitCode=0 Oct 30 18:41:47.133130 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.132942 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" Oct 30 18:41:47.133130 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.133012 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" event={"ID":"f8ad376f-2b5f-4907-8275-395d302442f8","Type":"ContainerDied","Data":"13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f"} Oct 30 18:41:47.133130 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.133053 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt" event={"ID":"f8ad376f-2b5f-4907-8275-395d302442f8","Type":"ContainerDied","Data":"7db1f59a5436da24cd88b87795a643fb910591c5f0f76be94d117befb3eb2182"} Oct 30 18:41:47.144425 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.144402 6160 scope.go:122] "RemoveContainer" containerID="f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8" Oct 30 18:41:47.144767 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:47.144748 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8\": container with ID starting with f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8 not found: ID does not exist" containerID="f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8" Oct 30 18:41:47.144848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.144773 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8"} err="failed to get container status \"f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8\": rpc error: code = NotFound desc = could not find container \"f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8\": container with ID starting with f7d4ec8e3e57a96bbfe088eddeef3df31f08649e16b942370df113a47adc3cf8 not found: ID does not exist" Oct 30 18:41:47.144848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.144790 6160 scope.go:122] "RemoveContainer" containerID="13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f" Oct 30 18:41:47.149097 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.149073 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume\") pod \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " Oct 30 18:41:47.149189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.149156 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68\" (UniqueName: \"kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68\") pod \"f8ad376f-2b5f-4907-8275-395d302442f8\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " Oct 30 18:41:47.149289 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.149260 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn\" (UniqueName: \"kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn\") pod \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\" (UID: \"57cb811d-cdf5-47aa-abf0-89b67b6403fd\") " Oct 30 18:41:47.149396 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.149324 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume\") pod \"f8ad376f-2b5f-4907-8275-395d302442f8\" (UID: \"f8ad376f-2b5f-4907-8275-395d302442f8\") " Oct 30 18:41:47.149475 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.149462 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume" pod "57cb811d-cdf5-47aa-abf0-89b67b6403fd" (UID: "57cb811d-cdf5-47aa-abf0-89b67b6403fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:47.149727 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.149704 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57cb811d-cdf5-47aa-abf0-89b67b6403fd-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:47.150100 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.150076 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume" pod "f8ad376f-2b5f-4907-8275-395d302442f8" (UID: "f8ad376f-2b5f-4907-8275-395d302442f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:47.151620 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.151600 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn" pod "57cb811d-cdf5-47aa-abf0-89b67b6403fd" (UID: "57cb811d-cdf5-47aa-abf0-89b67b6403fd"). InnerVolumeSpecName "kube-api-access-xdcrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:47.151697 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.151626 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68" pod "f8ad376f-2b5f-4907-8275-395d302442f8" (UID: "f8ad376f-2b5f-4907-8275-395d302442f8"). InnerVolumeSpecName "kube-api-access-stk68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:47.153475 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.153454 6160 scope.go:122] "RemoveContainer" containerID="13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f" Oct 30 18:41:47.153743 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:47.153720 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f\": container with ID starting with 13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f not found: ID does not exist" containerID="13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f" Oct 30 18:41:47.153814 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.153751 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f"} err="failed to get container status \"13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f\": rpc error: code = NotFound desc = could not find container \"13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f\": container with ID starting with 13337322af352d17474b55719bd9de529682ef859bd3e6f859223ea21ab96c3f not found: ID does not exist" Oct 30 18:41:47.250995 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.250931 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stk68\" (UniqueName: \"kubernetes.io/projected/f8ad376f-2b5f-4907-8275-395d302442f8-kube-api-access-stk68\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:47.250995 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.250995 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdcrn\" (UniqueName: \"kubernetes.io/projected/57cb811d-cdf5-47aa-abf0-89b67b6403fd-kube-api-access-xdcrn\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:47.250995 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.251012 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8ad376f-2b5f-4907-8275-395d302442f8-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:47.478062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.478030 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs"] Oct 30 18:41:47.483555 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.483532 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f7d8050f-9477df678-26mhs"] Oct 30 18:41:47.520411 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.520324 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt"] Oct 30 18:41:47.524926 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.524902 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f7d8050f-85c496447-vzmqt"] Oct 30 18:41:47.578275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.578241 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57cb811d-cdf5-47aa-abf0-89b67b6403fd" path="/var/lib/kubelet/pods/57cb811d-cdf5-47aa-abf0-89b67b6403fd/volumes" Oct 30 18:41:47.578623 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.578611 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e685408-7570-4784-9e34-0067f64326d0" path="/var/lib/kubelet/pods/5e685408-7570-4784-9e34-0067f64326d0/volumes" Oct 30 18:41:47.579000 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.578987 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d85ef0a-e693-4779-9ca7-57ff89a6ec96" path="/var/lib/kubelet/pods/6d85ef0a-e693-4779-9ca7-57ff89a6ec96/volumes" Oct 30 18:41:47.579632 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.579617 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd54e50c-3a59-4169-82cf-cff4ff3a7d07" path="/var/lib/kubelet/pods/dd54e50c-3a59-4169-82cf-cff4ff3a7d07/volumes" Oct 30 18:41:47.580020 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.580007 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8ad376f-2b5f-4907-8275-395d302442f8" path="/var/lib/kubelet/pods/f8ad376f-2b5f-4907-8275-395d302442f8/volumes" Oct 30 18:41:47.812751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.812716 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl"] Oct 30 18:41:47.813013 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:47.812943 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" podUID="13d4a624-4470-408a-9198-cfa19d88b744" containerName="maas-api" containerID="cri-o://37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2" gracePeriod=30 Oct 30 18:41:48.083729 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.083702 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:48.143888 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.143852 6160 generic.go:356] "Generic (PLEG): container finished" podID="13d4a624-4470-408a-9198-cfa19d88b744" containerID="37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2" exitCode=0 Oct 30 18:41:48.144076 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.143906 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" Oct 30 18:41:48.144076 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.143928 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" event={"ID":"13d4a624-4470-408a-9198-cfa19d88b744","Type":"ContainerDied","Data":"37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2"} Oct 30 18:41:48.144076 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.144023 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl" event={"ID":"13d4a624-4470-408a-9198-cfa19d88b744","Type":"ContainerDied","Data":"8c895e5a28e6d88fbfe3147edbf29a3fbb26b97676cfbe357d48784d9a9e0c71"} Oct 30 18:41:48.144076 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.144055 6160 scope.go:122] "RemoveContainer" containerID="37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2" Oct 30 18:41:48.158789 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.158770 6160 scope.go:122] "RemoveContainer" containerID="37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2" Oct 30 18:41:48.159137 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:48.159115 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2\": container with ID starting with 37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2 not found: ID does not exist" containerID="37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2" Oct 30 18:41:48.159213 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.159148 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2"} err="failed to get container status \"37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2\": rpc error: code = NotFound desc = could not find container \"37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2\": container with ID starting with 37347ff2bf3c645f124422c5a7b0846de5c82b5653d0eb60d9e6f42530e711b2 not found: ID does not exist" Oct 30 18:41:48.160471 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.160448 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca\") pod \"13d4a624-4470-408a-9198-cfa19d88b744\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " Oct 30 18:41:48.160597 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.160510 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls\") pod \"13d4a624-4470-408a-9198-cfa19d88b744\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " Oct 30 18:41:48.160660 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.160604 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs\") pod \"13d4a624-4470-408a-9198-cfa19d88b744\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " Oct 30 18:41:48.160660 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.160639 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6\" (UniqueName: \"kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6\") pod \"13d4a624-4470-408a-9198-cfa19d88b744\" (UID: \"13d4a624-4470-408a-9198-cfa19d88b744\") " Oct 30 18:41:48.160870 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.160845 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca" pod "13d4a624-4470-408a-9198-cfa19d88b744" (UID: "13d4a624-4470-408a-9198-cfa19d88b744"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:48.160999 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.160981 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d4a624-4470-408a-9198-cfa19d88b744-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.163003 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.162975 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6" pod "13d4a624-4470-408a-9198-cfa19d88b744" (UID: "13d4a624-4470-408a-9198-cfa19d88b744"). InnerVolumeSpecName "kube-api-access-84fc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:48.164982 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.163411 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs" pod "13d4a624-4470-408a-9198-cfa19d88b744" (UID: "13d4a624-4470-408a-9198-cfa19d88b744"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:48.164982 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.164099 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls" pod "13d4a624-4470-408a-9198-cfa19d88b744" (UID: "13d4a624-4470-408a-9198-cfa19d88b744"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:48.182472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.182444 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj"] Oct 30 18:41:48.182713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.182678 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" podUID="4e78b814-bb78-43e8-9c91-40ab995f77b8" containerName="maas-api" containerID="cri-o://1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3" gracePeriod=30 Oct 30 18:41:48.262378 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.262339 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.262378 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.262364 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13d4a624-4470-408a-9198-cfa19d88b744-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.262378 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.262375 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84fc6\" (UniqueName: \"kubernetes.io/projected/13d4a624-4470-408a-9198-cfa19d88b744-kube-api-access-84fc6\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.371208 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.371168 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:41:48.411281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.411255 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:48.464224 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.464190 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca\") pod \"4e78b814-bb78-43e8-9c91-40ab995f77b8\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " Oct 30 18:41:48.464396 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.464240 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\") pod \"4e78b814-bb78-43e8-9c91-40ab995f77b8\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " Oct 30 18:41:48.464396 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.464364 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7\" (UniqueName: \"kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7\") pod \"4e78b814-bb78-43e8-9c91-40ab995f77b8\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " Oct 30 18:41:48.464558 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.464416 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs\") pod \"4e78b814-bb78-43e8-9c91-40ab995f77b8\" (UID: \"4e78b814-bb78-43e8-9c91-40ab995f77b8\") " Oct 30 18:41:48.464644 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.464598 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca" pod "4e78b814-bb78-43e8-9c91-40ab995f77b8" (UID: "4e78b814-bb78-43e8-9c91-40ab995f77b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:48.464817 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.464797 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e78b814-bb78-43e8-9c91-40ab995f77b8-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.467007 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.466980 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7" pod "4e78b814-bb78-43e8-9c91-40ab995f77b8" (UID: "4e78b814-bb78-43e8-9c91-40ab995f77b8"). InnerVolumeSpecName "kube-api-access-7dmh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:48.467218 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.467198 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls" pod "4e78b814-bb78-43e8-9c91-40ab995f77b8" (UID: "4e78b814-bb78-43e8-9c91-40ab995f77b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:48.468586 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.468561 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs" pod "4e78b814-bb78-43e8-9c91-40ab995f77b8" (UID: "4e78b814-bb78-43e8-9c91-40ab995f77b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:48.497495 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.497466 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl"] Oct 30 18:41:48.504298 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.504271 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b77c9bba-7ff4b4fcb-c5mrl"] Oct 30 18:41:48.565484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.565459 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.565484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.565482 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dmh7\" (UniqueName: \"kubernetes.io/projected/4e78b814-bb78-43e8-9c91-40ab995f77b8-kube-api-access-7dmh7\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.565647 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.565491 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e78b814-bb78-43e8-9c91-40ab995f77b8-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:48.657276 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.657199 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb"] Oct 30 18:41:48.657429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.657399 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" podUID="bd22b256-c1c6-4040-8f56-b67e405a0782" containerName="maas-api" containerID="cri-o://c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5" gracePeriod=30 Oct 30 18:41:48.901889 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.901864 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:48.971570 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.971496 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls\") pod \"bd22b256-c1c6-4040-8f56-b67e405a0782\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " Oct 30 18:41:48.971713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.971588 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs\") pod \"bd22b256-c1c6-4040-8f56-b67e405a0782\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " Oct 30 18:41:48.971713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.971651 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca\") pod \"bd22b256-c1c6-4040-8f56-b67e405a0782\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " Oct 30 18:41:48.971813 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.971721 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4\" (UniqueName: \"kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4\") pod \"bd22b256-c1c6-4040-8f56-b67e405a0782\" (UID: \"bd22b256-c1c6-4040-8f56-b67e405a0782\") " Oct 30 18:41:48.972166 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.972134 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca" pod "bd22b256-c1c6-4040-8f56-b67e405a0782" (UID: "bd22b256-c1c6-4040-8f56-b67e405a0782"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:48.973643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.973613 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls" pod "bd22b256-c1c6-4040-8f56-b67e405a0782" (UID: "bd22b256-c1c6-4040-8f56-b67e405a0782"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:48.973830 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.973811 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4" pod "bd22b256-c1c6-4040-8f56-b67e405a0782" (UID: "bd22b256-c1c6-4040-8f56-b67e405a0782"). InnerVolumeSpecName "kube-api-access-mk7s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:48.974110 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:48.974091 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs" pod "bd22b256-c1c6-4040-8f56-b67e405a0782" (UID: "bd22b256-c1c6-4040-8f56-b67e405a0782"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:49.073461 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.073363 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk7s4\" (UniqueName: \"kubernetes.io/projected/bd22b256-c1c6-4040-8f56-b67e405a0782-kube-api-access-mk7s4\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:49.073461 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.073403 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:49.073461 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.073418 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd22b256-c1c6-4040-8f56-b67e405a0782-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:49.073461 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.073429 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd22b256-c1c6-4040-8f56-b67e405a0782-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:49.155657 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.155625 6160 generic.go:356] "Generic (PLEG): container finished" podID="bd22b256-c1c6-4040-8f56-b67e405a0782" containerID="c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5" exitCode=0 Oct 30 18:41:49.156128 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.155694 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" Oct 30 18:41:49.156128 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.155709 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" event={"ID":"bd22b256-c1c6-4040-8f56-b67e405a0782","Type":"ContainerDied","Data":"c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5"} Oct 30 18:41:49.156128 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.155762 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb" event={"ID":"bd22b256-c1c6-4040-8f56-b67e405a0782","Type":"ContainerDied","Data":"7b6d3679523002d2ef45bb833645d7306c26da51d0091412df5a8605d0ae846c"} Oct 30 18:41:49.156128 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.155787 6160 scope.go:122] "RemoveContainer" containerID="c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5" Oct 30 18:41:49.157472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.157450 6160 generic.go:356] "Generic (PLEG): container finished" podID="4e78b814-bb78-43e8-9c91-40ab995f77b8" containerID="1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3" exitCode=0 Oct 30 18:41:49.157709 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.157506 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" Oct 30 18:41:49.157709 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.157540 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" event={"ID":"4e78b814-bb78-43e8-9c91-40ab995f77b8","Type":"ContainerDied","Data":"1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3"} Oct 30 18:41:49.157709 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.157600 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj" event={"ID":"4e78b814-bb78-43e8-9c91-40ab995f77b8","Type":"ContainerDied","Data":"4452e9b3b3faacafd15fd180212c6fed083b567d33a7fd25ae8e293acb5218f3"} Oct 30 18:41:49.165284 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.165261 6160 scope.go:122] "RemoveContainer" containerID="c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5" Oct 30 18:41:49.165578 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:49.165556 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5\": container with ID starting with c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5 not found: ID does not exist" containerID="c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5" Oct 30 18:41:49.165660 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.165589 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5"} err="failed to get container status \"c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5\": rpc error: code = NotFound desc = could not find container \"c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5\": container with ID starting with c4c55664ed353bb5a86d81f0d433e23e054366268e8938c6501eadf348f7bde5 not found: ID does not exist" Oct 30 18:41:49.165660 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.165616 6160 scope.go:122] "RemoveContainer" containerID="1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3" Oct 30 18:41:49.174816 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.174800 6160 scope.go:122] "RemoveContainer" containerID="1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3" Oct 30 18:41:49.175102 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:49.175081 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3\": container with ID starting with 1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3 not found: ID does not exist" containerID="1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3" Oct 30 18:41:49.175162 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.175107 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3"} err="failed to get container status \"1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3\": rpc error: code = NotFound desc = could not find container \"1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3\": container with ID starting with 1757a687c1200c671dcc1fde90b92fd3ae10806655f0c37ff97b26c0cd0178a3 not found: ID does not exist" Oct 30 18:41:49.204734 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.204706 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb"] Oct 30 18:41:49.209281 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.209259 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed302a08-878cf8b89-96vfb"] Oct 30 18:41:49.242827 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.242746 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj"] Oct 30 18:41:49.250275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.250245 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48afdd9e-9dfc67bc5-g65sj"] Oct 30 18:41:49.425432 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.425404 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd"] Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426079 6160 cpu_manager.go:414] "containerMap: removing container" podUID="5e685408-7570-4784-9e34-0067f64326d0" containerName="payload-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426105 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e685408-7570-4784-9e34-0067f64326d0" containerName="payload-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426159 6160 cpu_manager.go:414] "containerMap: removing container" podUID="dd54e50c-3a59-4169-82cf-cff4ff3a7d07" containerName="payload-pre-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426168 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd54e50c-3a59-4169-82cf-cff4ff3a7d07" containerName="payload-pre-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426181 6160 cpu_manager.go:414] "containerMap: removing container" podUID="6d85ef0a-e693-4779-9ca7-57ff89a6ec96" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426189 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d85ef0a-e693-4779-9ca7-57ff89a6ec96" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426205 6160 cpu_manager.go:414] "containerMap: removing container" podUID="bd22b256-c1c6-4040-8f56-b67e405a0782" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426211 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd22b256-c1c6-4040-8f56-b67e405a0782" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426222 6160 cpu_manager.go:414] "containerMap: removing container" podUID="13d4a624-4470-408a-9198-cfa19d88b744" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426231 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13d4a624-4470-408a-9198-cfa19d88b744" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426242 6160 cpu_manager.go:414] "containerMap: removing container" podUID="4e78b814-bb78-43e8-9c91-40ab995f77b8" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426249 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e78b814-bb78-43e8-9c91-40ab995f77b8" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426263 6160 cpu_manager.go:414] "containerMap: removing container" podUID="f8ad376f-2b5f-4907-8275-395d302442f8" containerName="payload-pre-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426268 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8ad376f-2b5f-4907-8275-395d302442f8" containerName="payload-pre-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426275 6160 cpu_manager.go:414] "containerMap: removing container" podUID="57cb811d-cdf5-47aa-abf0-89b67b6403fd" containerName="payload-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426281 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57cb811d-cdf5-47aa-abf0-89b67b6403fd" containerName="payload-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426355 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd54e50c-3a59-4169-82cf-cff4ff3a7d07" containerName="payload-pre-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426363 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d85ef0a-e693-4779-9ca7-57ff89a6ec96" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426369 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd22b256-c1c6-4040-8f56-b67e405a0782" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426375 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="13d4a624-4470-408a-9198-cfa19d88b744" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426381 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e78b814-bb78-43e8-9c91-40ab995f77b8" containerName="maas-api" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426387 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8ad376f-2b5f-4907-8275-395d302442f8" containerName="payload-pre-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426393 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="57cb811d-cdf5-47aa-abf0-89b67b6403fd" containerName="payload-processing" Oct 30 18:41:49.426983 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.426400 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e685408-7570-4784-9e34-0067f64326d0" containerName="payload-processing" Oct 30 18:41:49.430216 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.430197 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.446161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.446137 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-a47385-openshift-default-dockercfg-p98h9\"" Oct 30 18:41:49.451683 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.451651 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd"] Oct 30 18:41:49.582482 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582450 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582663 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582494 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582663 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582586 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582663 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582644 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582798 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582675 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnt75\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582798 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582746 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582798 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582791 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582824 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.582907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.582855 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.586450 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.586386 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13d4a624-4470-408a-9198-cfa19d88b744" path="/var/lib/kubelet/pods/13d4a624-4470-408a-9198-cfa19d88b744/volumes" Oct 30 18:41:49.586823 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.586809 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e78b814-bb78-43e8-9c91-40ab995f77b8" path="/var/lib/kubelet/pods/4e78b814-bb78-43e8-9c91-40ab995f77b8/volumes" Oct 30 18:41:49.587192 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.587179 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd22b256-c1c6-4040-8f56-b67e405a0782" path="/var/lib/kubelet/pods/bd22b256-c1c6-4040-8f56-b67e405a0782/volumes" Oct 30 18:41:49.683550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.683522 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.683714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.683573 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.683714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.683598 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.683714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.683641 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.683714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.683674 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.683714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.683700 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnt75\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684063 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684037 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684077 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684130 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684166 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684378 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684479 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684399 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684560 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684536 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.684618 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.684600 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.686189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.686167 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.686320 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.686297 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.707574 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.707547 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnt75\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.708758 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.708737 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token\") pod \"e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:49.741886 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:49.741858 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:50.072384 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.072099 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd"] Oct 30 18:41:50.076741 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:50.076715 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c2faa6_6f47_4ed2_bd5f_1d0eb97943f8.slice/crio-c8cd390f0e7d5ee106ea92f42e8332ca4f6e34730db05fe642791fc5feb84fbb WatchSource:0}: Error finding container c8cd390f0e7d5ee106ea92f42e8332ca4f6e34730db05fe642791fc5feb84fbb: Status 404 returned error can't find the container with id c8cd390f0e7d5ee106ea92f42e8332ca4f6e34730db05fe642791fc5feb84fbb Oct 30 18:41:50.078936 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.078904 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:50.079037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.078983 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:50.079037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.079014 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:50.164708 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.164641 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt"] Oct 30 18:41:50.168323 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.168297 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.168843 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.168813 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" event={"ID":"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8","Type":"ContainerStarted","Data":"a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093"} Oct 30 18:41:50.169058 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.169026 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" event={"ID":"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8","Type":"ContainerStarted","Data":"c8cd390f0e7d5ee106ea92f42e8332ca4f6e34730db05fe642791fc5feb84fbb"} Oct 30 18:41:50.183451 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.183427 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-0ddc26-openshift-default-dockercfg-sc7gm\"" Oct 30 18:41:50.197638 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.197608 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt"] Oct 30 18:41:50.288552 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.288517 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.288809 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.288787 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.288976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.288943 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.289124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.289106 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.289259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.289242 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.289371 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.289356 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.289525 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.289511 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6jcr\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.289660 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.289605 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.289797 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.289781 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391155 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391202 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391298 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391320 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391344 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391374 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391398 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391423 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391474 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6jcr\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391666 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.391970 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.391930 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.392084 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.392055 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.392199 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.392177 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.392482 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.392442 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.394299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.394273 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.394406 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.394318 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.415645 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.415604 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6jcr\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.420057 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.419457 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.483659 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.483621 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:50.638575 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.637798 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podStartSLOduration=1.637778167 podStartE2EDuration="1.637778167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:50.268721821 +0000 UTC m=+580.806914028" watchObservedRunningTime="2026-10-30 18:41:50.637778167 +0000 UTC m=+581.175970373" Oct 30 18:41:50.642037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.639474 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn"] Oct 30 18:41:50.642474 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.642457 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.657181 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.657156 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-b2ef42-openshift-default-dockercfg-m5wtg\"" Oct 30 18:41:50.669181 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.669141 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn"] Oct 30 18:41:50.742484 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.742458 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:50.796429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796399 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796436 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796622 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796463 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796622 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796557 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796633 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796676 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796699 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796832 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796786 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.796832 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.796824 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnpsd\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.803672 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.803638 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt"] Oct 30 18:41:50.809341 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:50.809313 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b0a55b_e180_49d3_a519_c79eabd7d6de.slice/crio-26dc2232a8527053820b423e11584353a34709989fc2d27233a7671e4542dea4 WatchSource:0}: Error finding container 26dc2232a8527053820b423e11584353a34709989fc2d27233a7671e4542dea4: Status 404 returned error can't find the container with id 26dc2232a8527053820b423e11584353a34709989fc2d27233a7671e4542dea4 Oct 30 18:41:50.812126 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.812094 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:50.812212 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.812173 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:50.812212 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.812201 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.904645 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.904743 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.904807 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.904835 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.904939 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.905062 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnpsd\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.905139 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.905191 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.905234 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.905822 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.906228 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.906884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.906443 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.909708 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.909048 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.909708 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.909434 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.909708 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.909667 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.922891 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.922855 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.934385 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.934353 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnpsd\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.934855 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.934834 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token\") pod \"e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:50.958218 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:50.958183 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:51.178545 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.178452 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" event={"ID":"c5b0a55b-e180-49d3-a519-c79eabd7d6de","Type":"ContainerStarted","Data":"715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee"} Oct 30 18:41:51.178545 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.178504 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" event={"ID":"c5b0a55b-e180-49d3-a519-c79eabd7d6de","Type":"ContainerStarted","Data":"26dc2232a8527053820b423e11584353a34709989fc2d27233a7671e4542dea4"} Oct 30 18:41:51.231505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.231452 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podStartSLOduration=1.231434254 podStartE2EDuration="1.231434254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:51.230574315 +0000 UTC m=+581.768766514" watchObservedRunningTime="2026-10-30 18:41:51.231434254 +0000 UTC m=+581.769626459" Oct 30 18:41:51.271996 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.271947 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn"] Oct 30 18:41:51.275717 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:51.275694 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1cb278_d548_45cd_8919_cc38e0be4554.slice/crio-8e0c3262645d7cb70699263a6f536ed8511e96ab11c5ac5e1a8a70ee6515eaaa WatchSource:0}: Error finding container 8e0c3262645d7cb70699263a6f536ed8511e96ab11c5ac5e1a8a70ee6515eaaa: Status 404 returned error can't find the container with id 8e0c3262645d7cb70699263a6f536ed8511e96ab11c5ac5e1a8a70ee6515eaaa Oct 30 18:41:51.278435 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.278406 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:51.278504 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.278470 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:51.278542 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.278502 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:51.484574 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.484496 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:51.490307 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.490279 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:51.591902 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.591863 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" podUID="4eac19ae-a9a2-4685-b724-52512b4c5de0" containerName="payload-processing" containerID="cri-o://9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831" gracePeriod=25 Oct 30 18:41:51.624145 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.624089 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" podUID="ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" containerName="payload-pre-processing" containerID="cri-o://baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05" gracePeriod=25 Oct 30 18:41:51.696140 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:51.696109 6160 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae98a480_a3c6_4131_85b4_cbb77e1dc2d7.slice/crio-conmon-baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05.scope\": RecentStats: unable to find data in memory cache]" Oct 30 18:41:51.749559 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.749528 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:51.883561 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.883536 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:51.897884 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:51.897862 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:52.017748 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.017667 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume\") pod \"4eac19ae-a9a2-4685-b724-52512b4c5de0\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " Oct 30 18:41:52.017904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.017756 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s\" (UniqueName: \"kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s\") pod \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " Oct 30 18:41:52.017904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.017807 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7\" (UniqueName: \"kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7\") pod \"4eac19ae-a9a2-4685-b724-52512b4c5de0\" (UID: \"4eac19ae-a9a2-4685-b724-52512b4c5de0\") " Oct 30 18:41:52.017904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.017856 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume\") pod \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\" (UID: \"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7\") " Oct 30 18:41:52.018265 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.018239 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume" pod "4eac19ae-a9a2-4685-b724-52512b4c5de0" (UID: "4eac19ae-a9a2-4685-b724-52512b4c5de0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:52.018345 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.018261 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume" pod "ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" (UID: "ae98a480-a3c6-4131-85b4-cbb77e1dc2d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:52.019937 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.019916 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7" pod "4eac19ae-a9a2-4685-b724-52512b4c5de0" (UID: "4eac19ae-a9a2-4685-b724-52512b4c5de0"). InnerVolumeSpecName "kube-api-access-pjfn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:52.020005 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.019964 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s" pod "ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" (UID: "ae98a480-a3c6-4131-85b4-cbb77e1dc2d7"). InnerVolumeSpecName "kube-api-access-zzs6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:52.119106 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.119065 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac19ae-a9a2-4685-b724-52512b4c5de0-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:52.119106 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.119103 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzs6s\" (UniqueName: \"kubernetes.io/projected/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-kube-api-access-zzs6s\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:52.119321 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.119120 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjfn7\" (UniqueName: \"kubernetes.io/projected/4eac19ae-a9a2-4685-b724-52512b4c5de0-kube-api-access-pjfn7\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:52.119321 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.119133 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:52.185355 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.185323 6160 generic.go:356] "Generic (PLEG): container finished" podID="ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" containerID="baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05" exitCode=0 Oct 30 18:41:52.185784 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.185368 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" Oct 30 18:41:52.185784 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.185404 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" event={"ID":"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7","Type":"ContainerDied","Data":"baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05"} Oct 30 18:41:52.185784 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.185463 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk" event={"ID":"ae98a480-a3c6-4131-85b4-cbb77e1dc2d7","Type":"ContainerDied","Data":"eb178bac2d21b73787c5a8e40d7fd0c7773346b0047728269e3f8d53a6db0a19"} Oct 30 18:41:52.185784 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.185493 6160 scope.go:122] "RemoveContainer" containerID="baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05" Oct 30 18:41:52.187566 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.187536 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" event={"ID":"fb1cb278-d548-45cd-8919-cc38e0be4554","Type":"ContainerStarted","Data":"fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd"} Oct 30 18:41:52.187688 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.187588 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" event={"ID":"fb1cb278-d548-45cd-8919-cc38e0be4554","Type":"ContainerStarted","Data":"8e0c3262645d7cb70699263a6f536ed8511e96ab11c5ac5e1a8a70ee6515eaaa"} Oct 30 18:41:52.189648 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.189622 6160 generic.go:356] "Generic (PLEG): container finished" podID="4eac19ae-a9a2-4685-b724-52512b4c5de0" containerID="9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831" exitCode=0 Oct 30 18:41:52.189750 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.189676 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" Oct 30 18:41:52.189750 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.189707 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" event={"ID":"4eac19ae-a9a2-4685-b724-52512b4c5de0","Type":"ContainerDied","Data":"9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831"} Oct 30 18:41:52.189871 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.189768 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr" event={"ID":"4eac19ae-a9a2-4685-b724-52512b4c5de0","Type":"ContainerDied","Data":"e091b87599b61adec5a8c521dd694c503e24a5ce4b4efa052a45e253b83db88b"} Oct 30 18:41:52.190768 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.190748 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:52.191443 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.191424 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:52.192941 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.192697 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:41:52.192941 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.192850 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:41:52.196224 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.196201 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf"] Oct 30 18:41:52.196713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.196698 6160 scope.go:122] "RemoveContainer" containerID="baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05" Oct 30 18:41:52.196836 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.196815 6160 cpu_manager.go:414] "containerMap: removing container" podUID="ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" containerName="payload-pre-processing" Oct 30 18:41:52.196989 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.196967 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" containerName="payload-pre-processing" Oct 30 18:41:52.197084 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.197073 6160 cpu_manager.go:414] "containerMap: removing container" podUID="4eac19ae-a9a2-4685-b724-52512b4c5de0" containerName="payload-processing" Oct 30 18:41:52.197158 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.197147 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4eac19ae-a9a2-4685-b724-52512b4c5de0" containerName="payload-processing" Oct 30 18:41:52.197356 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:52.197091 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05\": container with ID starting with baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05 not found: ID does not exist" containerID="baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05" Oct 30 18:41:52.197441 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.197373 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05"} err="failed to get container status \"baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05\": rpc error: code = NotFound desc = could not find container \"baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05\": container with ID starting with baac81ab97330dec680e8eafc936f368abee59ac0313628274b5197c3a0cca05 not found: ID does not exist" Oct 30 18:41:52.197441 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.197396 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" containerName="payload-pre-processing" Oct 30 18:41:52.197441 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.197413 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="4eac19ae-a9a2-4685-b724-52512b4c5de0" containerName="payload-processing" Oct 30 18:41:52.197539 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.197400 6160 scope.go:122] "RemoveContainer" containerID="9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831" Oct 30 18:41:52.200304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.200285 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.209500 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.209410 6160 scope.go:122] "RemoveContainer" containerID="9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831" Oct 30 18:41:52.209758 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:52.209733 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831\": container with ID starting with 9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831 not found: ID does not exist" containerID="9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831" Oct 30 18:41:52.209838 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.209765 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831"} err="failed to get container status \"9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831\": rpc error: code = NotFound desc = could not find container \"9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831\": container with ID starting with 9eff399477a2370686c31a29fa7586a3791891b3751de065a8db63523b5db831 not found: ID does not exist" Oct 30 18:41:52.217402 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.217385 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f6f7d5-openshift-default-dockercfg-9sw8n\"" Oct 30 18:41:52.225771 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.225748 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf"] Oct 30 18:41:52.300937 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.300883 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podStartSLOduration=2.300866172 podStartE2EDuration="2.300866172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:52.30002989 +0000 UTC m=+582.838222094" watchObservedRunningTime="2026-10-30 18:41:52.300866172 +0000 UTC m=+582.839058379" Oct 30 18:41:52.320806 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.320775 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.320976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.320815 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321099 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321079 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321292 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321262 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321313 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321558 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321534 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321772 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321752 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321843 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321791 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn54h\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.321880 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.321835 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.392316 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.392280 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk"] Oct 30 18:41:52.398946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.398911 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48afdd9e-6b954754c-sx2qk"] Oct 30 18:41:52.423137 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423105 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423165 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423198 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn54h\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423233 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423311 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423336 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423362 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423414 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423439 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423746 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423696 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.423834 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423745 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.424014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.423993 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.424167 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.424147 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.424263 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.424242 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.425589 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.425569 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.425649 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.425634 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.446302 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.446272 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.447419 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.447398 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn54h\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.495980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.493769 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr"] Oct 30 18:41:52.501979 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.500351 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48afdd9e-7d9796f4dd-s76rr"] Oct 30 18:41:52.518336 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.518304 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:52.829933 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.829822 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf"] Oct 30 18:41:52.833486 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:41:52.833454 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7aee38f_d8ec_4af1_b514_8ee63c0b1b96.slice/crio-fc1fa702d184bbf43a16771a2ba274d8da1f0689293a481c53c5e010c27805b2 WatchSource:0}: Error finding container fc1fa702d184bbf43a16771a2ba274d8da1f0689293a481c53c5e010c27805b2: Status 404 returned error can't find the container with id fc1fa702d184bbf43a16771a2ba274d8da1f0689293a481c53c5e010c27805b2 Oct 30 18:41:52.835810 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.835774 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:52.835896 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.835843 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:52.835896 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.835872 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:41:52.959462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.959110 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:52.964583 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:52.964556 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:53.196358 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.196260 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" event={"ID":"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96","Type":"ContainerStarted","Data":"a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d"} Oct 30 18:41:53.196358 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.196328 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" event={"ID":"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96","Type":"ContainerStarted","Data":"fc1fa702d184bbf43a16771a2ba274d8da1f0689293a481c53c5e010c27805b2"} Oct 30 18:41:53.198496 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.198463 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:53.199398 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.199381 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:41:53.252382 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.252328 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podStartSLOduration=1.252309356 podStartE2EDuration="1.252309356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:53.251605172 +0000 UTC m=+583.789797377" watchObservedRunningTime="2026-10-30 18:41:53.252309356 +0000 UTC m=+583.790501564" Oct 30 18:41:53.538714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.535509 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:53.545034 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.544998 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:53.588736 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.588692 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4eac19ae-a9a2-4685-b724-52512b4c5de0" path="/var/lib/kubelet/pods/4eac19ae-a9a2-4685-b724-52512b4c5de0/volumes" Oct 30 18:41:53.589210 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:53.589192 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae98a480-a3c6-4131-85b4-cbb77e1dc2d7" path="/var/lib/kubelet/pods/ae98a480-a3c6-4131-85b4-cbb77e1dc2d7/volumes" Oct 30 18:41:54.204512 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:54.204477 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:54.205011 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:54.204993 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:41:55.254081 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.236444 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw"] Oct 30 18:41:55.254081 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.236728 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" podUID="da70d7f5-08f8-48ec-ad43-3926eb875ef4" containerName="maas-api" containerID="cri-o://3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410" gracePeriod=30 Oct 30 18:41:55.508123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.508055 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:55.559186 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.559160 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca\") pod \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " Oct 30 18:41:55.559364 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.559231 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb\" (UniqueName: \"kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb\") pod \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " Oct 30 18:41:55.559364 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.559334 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls\") pod \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " Oct 30 18:41:55.559494 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.559418 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs\") pod \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\" (UID: \"da70d7f5-08f8-48ec-ad43-3926eb875ef4\") " Oct 30 18:41:55.559600 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.559578 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca" pod "da70d7f5-08f8-48ec-ad43-3926eb875ef4" (UID: "da70d7f5-08f8-48ec-ad43-3926eb875ef4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:55.559870 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.559842 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da70d7f5-08f8-48ec-ad43-3926eb875ef4-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:55.561655 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.561628 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb" pod "da70d7f5-08f8-48ec-ad43-3926eb875ef4" (UID: "da70d7f5-08f8-48ec-ad43-3926eb875ef4"). InnerVolumeSpecName "kube-api-access-v4kkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:55.561820 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.561801 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs" pod "da70d7f5-08f8-48ec-ad43-3926eb875ef4" (UID: "da70d7f5-08f8-48ec-ad43-3926eb875ef4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:55.565259 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.565234 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls" pod "da70d7f5-08f8-48ec-ad43-3926eb875ef4" (UID: "da70d7f5-08f8-48ec-ad43-3926eb875ef4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:55.661490 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.661406 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4kkb\" (UniqueName: \"kubernetes.io/projected/da70d7f5-08f8-48ec-ad43-3926eb875ef4-kube-api-access-v4kkb\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:55.661490 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.661444 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:55.661490 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.661461 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da70d7f5-08f8-48ec-ad43-3926eb875ef4-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:41:55.892790 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.892753 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch"] Oct 30 18:41:55.893312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.893285 6160 cpu_manager.go:414] "containerMap: removing container" podUID="da70d7f5-08f8-48ec-ad43-3926eb875ef4" containerName="maas-api" Oct 30 18:41:55.893312 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.893305 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da70d7f5-08f8-48ec-ad43-3926eb875ef4" containerName="maas-api" Oct 30 18:41:55.893439 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.893400 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="da70d7f5-08f8-48ec-ad43-3926eb875ef4" containerName="maas-api" Oct 30 18:41:55.895913 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.895894 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:55.912370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.912349 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-a47385-serving-cert\"" Oct 30 18:41:55.918829 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.918808 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch"] Oct 30 18:41:55.964649 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.964618 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:55.964764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.964727 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:55.964764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.964753 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:55.964842 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:55.964798 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxrnc\" (UniqueName: \"kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.065387 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.065361 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.065546 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.065398 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.065546 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.065468 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxrnc\" (UniqueName: \"kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.065546 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:56.065530 6160 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-serving-cert: secret "maas-api-e2e-worker-w4-a47385-serving-cert" not found Oct 30 18:41:56.065720 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.065560 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.065720 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:56.065585 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls podName:8ef936eb-defc-410f-9d50-7392e6ad2381 nodeName:}" failed. No retries permitted until 2026-10-30 18:41:56.56556918 +0000 UTC m=+587.103761376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls") pod "maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" (UID: "8ef936eb-defc-410f-9d50-7392e6ad2381") : secret "maas-api-e2e-worker-w4-a47385-serving-cert" not found Oct 30 18:41:56.066116 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.066096 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.068003 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.067984 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.097310 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.097287 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxrnc\" (UniqueName: \"kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.206692 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.206618 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq"] Oct 30 18:41:56.206888 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.206847 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" containerID="cri-o://8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504" gracePeriod=30 Oct 30 18:41:56.225314 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.225283 6160 generic.go:356] "Generic (PLEG): container finished" podID="da70d7f5-08f8-48ec-ad43-3926eb875ef4" containerID="3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410" exitCode=0 Oct 30 18:41:56.225462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.225334 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" Oct 30 18:41:56.225462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.225371 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" event={"ID":"da70d7f5-08f8-48ec-ad43-3926eb875ef4","Type":"ContainerDied","Data":"3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410"} Oct 30 18:41:56.225462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.225426 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw" event={"ID":"da70d7f5-08f8-48ec-ad43-3926eb875ef4","Type":"ContainerDied","Data":"fbbb0a9cc354a144a96c2ba1a22a02a50257e728448dd0df23d914b225a37fb4"} Oct 30 18:41:56.225462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.225456 6160 scope.go:122] "RemoveContainer" containerID="3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410" Oct 30 18:41:56.234406 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.234390 6160 scope.go:122] "RemoveContainer" containerID="3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410" Oct 30 18:41:56.234693 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:41:56.234667 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410\": container with ID starting with 3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410 not found: ID does not exist" containerID="3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410" Oct 30 18:41:56.234780 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.234693 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410"} err="failed to get container status \"3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410\": rpc error: code = NotFound desc = could not find container \"3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410\": container with ID starting with 3e13b1f7bffc1b94f7aa68ba8930998429a3d995a11611d17990b4247d35a410 not found: ID does not exist" Oct 30 18:41:56.285595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.285570 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw"] Oct 30 18:41:56.290433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.290412 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a33e5151-85cfd7649-94mqw"] Oct 30 18:41:56.572839 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.572803 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.575307 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.575277 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\") pod \"maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:56.808122 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:56.808089 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:57.113400 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:57.113365 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch"] Oct 30 18:41:57.232944 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:57.232907 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" event={"ID":"8ef936eb-defc-410f-9d50-7392e6ad2381","Type":"ContainerStarted","Data":"b1c380c796640ada490c89a0419e6adf583763fd09cfce6d9489e9a3ba3fe178"} Oct 30 18:41:57.578496 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:57.578461 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da70d7f5-08f8-48ec-ad43-3926eb875ef4" path="/var/lib/kubelet/pods/da70d7f5-08f8-48ec-ad43-3926eb875ef4/volumes" Oct 30 18:41:58.240444 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:58.240413 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" event={"ID":"8ef936eb-defc-410f-9d50-7392e6ad2381","Type":"ContainerStarted","Data":"94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85"} Oct 30 18:41:58.240635 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:58.240543 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:41:58.284116 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:58.284055 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" podStartSLOduration=2.915936396 podStartE2EDuration="3.284021867s" totalImagesPullingTime="368.085471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:58.283374489 +0000 UTC m=+588.821566694" watchObservedRunningTime="2026-10-30 18:41:58.284021867 +0000 UTC m=+588.822214071" Oct 30 18:41:58.858918 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:41:58.858884 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:42:01.504757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.504727 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:42:01.644551 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644517 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.644551 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644556 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.644774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644602 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.644774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644621 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.644774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644655 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.644774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644740 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.645005 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644800 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.645005 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644840 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.645005 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.644918 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.645753 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.646069 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.646356 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.646416 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq\") pod \"99097643-a6bb-46f4-8804-9b10d8317a28\" (UID: \"99097643-a6bb-46f4-8804-9b10d8317a28\") " Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.646944 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.647226 6160 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.647259 6160 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-data\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.647273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.647274 6160 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99097643-a6bb-46f4-8804-9b10d8317a28-istiod-ca-cert\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.647643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.647287 6160 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-workload-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.647643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.647304 6160 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-credential-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.670973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.648293 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:42:01.686487 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.686442 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:42:01.686619 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.686494 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:42:01.687764 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.687723 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq" pod "99097643-a6bb-46f4-8804-9b10d8317a28" (UID: "99097643-a6bb-46f4-8804-9b10d8317a28"). InnerVolumeSpecName "kube-api-access-q4lmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:42:01.750185 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.750092 6160 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99097643-a6bb-46f4-8804-9b10d8317a28-istio-podinfo\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.750524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.750502 6160 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99097643-a6bb-46f4-8804-9b10d8317a28-istio-envoy\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.750673 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.750653 6160 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-istio-token\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.750790 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.750775 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4lmq\" (UniqueName: \"kubernetes.io/projected/99097643-a6bb-46f4-8804-9b10d8317a28-kube-api-access-q4lmq\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:42:01.936020 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.935975 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v"] Oct 30 18:42:01.936528 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.936511 6160 cpu_manager.go:414] "containerMap: removing container" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" Oct 30 18:42:01.936588 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.936529 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" Oct 30 18:42:01.936588 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.936554 6160 cpu_manager.go:414] "containerMap: removing container" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" Oct 30 18:42:01.936588 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.936559 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" Oct 30 18:42:01.936694 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.936623 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" Oct 30 18:42:01.936694 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.936633 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" containerName="istio-proxy" Oct 30 18:42:01.939004 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.938986 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:01.980980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.973054 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-b2ef42-serving-cert\"" Oct 30 18:42:01.980980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:01.980830 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v"] Oct 30 18:42:02.074980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.074082 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.074980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.074134 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.074980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.074231 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlj96\" (UniqueName: \"kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.074980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.074286 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.178456 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.178424 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.178456 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.178459 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.178706 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.178527 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlj96\" (UniqueName: \"kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.178706 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.178569 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.179363 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.179332 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.181703 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.181678 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.182147 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.182126 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.210306 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.209807 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlj96\" (UniqueName: \"kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96\") pod \"maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.251431 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.251403 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:02.304971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.304484 6160 generic.go:356] "Generic (PLEG): container finished" podID="99097643-a6bb-46f4-8804-9b10d8317a28" containerID="8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504" exitCode=0 Oct 30 18:42:02.304971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.304647 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" event={"ID":"99097643-a6bb-46f4-8804-9b10d8317a28","Type":"ContainerDied","Data":"8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504"} Oct 30 18:42:02.304971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.304699 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" event={"ID":"99097643-a6bb-46f4-8804-9b10d8317a28","Type":"ContainerDied","Data":"0665bf358cc1735c3b259e82703fb44e087e766a353bcacadc2f357e1b89b27c"} Oct 30 18:42:02.304971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.304729 6160 scope.go:122] "RemoveContainer" containerID="8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504" Oct 30 18:42:02.304971 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.304906 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq" Oct 30 18:42:02.317276 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.317256 6160 scope.go:122] "RemoveContainer" containerID="5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58" Oct 30 18:42:02.328737 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.328555 6160 scope.go:122] "RemoveContainer" containerID="8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504" Oct 30 18:42:02.328912 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:42:02.328884 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504\": container with ID starting with 8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504 not found: ID does not exist" containerID="8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504" Oct 30 18:42:02.329388 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.328925 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504"} err="failed to get container status \"8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504\": rpc error: code = NotFound desc = could not find container \"8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504\": container with ID starting with 8308b2a7e72686817b6091d3e7be67eced3a1c979d04dd106a291d11b6dca504 not found: ID does not exist" Oct 30 18:42:02.329388 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.328964 6160 scope.go:122] "RemoveContainer" containerID="5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58" Oct 30 18:42:02.329567 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:42:02.329462 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58\": container with ID starting with 5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58 not found: ID does not exist" containerID="5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58" Oct 30 18:42:02.329567 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.329497 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58"} err="failed to get container status \"5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58\": rpc error: code = NotFound desc = could not find container \"5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58\": container with ID starting with 5ee465c46f212acb1d7f7350571793df3ad1e587101dbf1285c063c040b37e58 not found: ID does not exist" Oct 30 18:42:02.381582 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.381500 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq"] Oct 30 18:42:02.383074 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.383046 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a33e5151-openshift-default-7ff8bf87cc-7xsfq"] Oct 30 18:42:02.521233 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.521193 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:42:02.635280 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:02.635181 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v"] Oct 30 18:42:02.639311 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:42:02.639285 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d213625_ac20_4e8b_8532_f69d9efd1f83.slice/crio-c655e62b887f929b0236537546bde2d5c0352ab4300dd92d4e9c8cad018f102b WatchSource:0}: Error finding container c655e62b887f929b0236537546bde2d5c0352ab4300dd92d4e9c8cad018f102b: Status 404 returned error can't find the container with id c655e62b887f929b0236537546bde2d5c0352ab4300dd92d4e9c8cad018f102b Oct 30 18:42:03.313761 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:03.313723 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" event={"ID":"0d213625-ac20-4e8b-8532-f69d9efd1f83","Type":"ContainerStarted","Data":"ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f"} Oct 30 18:42:03.313980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:03.313773 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" event={"ID":"0d213625-ac20-4e8b-8532-f69d9efd1f83","Type":"ContainerStarted","Data":"c655e62b887f929b0236537546bde2d5c0352ab4300dd92d4e9c8cad018f102b"} Oct 30 18:42:03.313980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:03.313834 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:03.611300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:03.611109 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99097643-a6bb-46f4-8804-9b10d8317a28" path="/var/lib/kubelet/pods/99097643-a6bb-46f4-8804-9b10d8317a28/volumes" Oct 30 18:42:04.286980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.283143 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:42:04.339333 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.339269 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" podStartSLOduration=2.960919092 podStartE2EDuration="3.339247694s" totalImagesPullingTime="378.328602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:03.374452507 +0000 UTC m=+593.912644713" watchObservedRunningTime="2026-10-30 18:42:04.339247694 +0000 UTC m=+594.877439876" Oct 30 18:42:04.575466 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.575378 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg"] Oct 30 18:42:04.615757 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.615729 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.617177 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.615636 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg"] Oct 30 18:42:04.636973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.636587 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-a759a1-dockercfg-fwcsl\"" Oct 30 18:42:04.636973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.636675 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-a759a1\"" Oct 30 18:42:04.735876 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.735175 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-254nz\" (UniqueName: \"kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz\") pod \"payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.735876 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.735451 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.836300 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.836215 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-254nz\" (UniqueName: \"kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz\") pod \"payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.836448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.836322 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.837040 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.837020 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.904104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.904070 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-254nz\" (UniqueName: \"kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz\") pod \"payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:04.946138 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:04.946111 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:05.306973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:05.306915 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg"] Oct 30 18:42:05.323814 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:42:05.323774 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ae66cf_3dc0_451a_a5be_1f2b2e057465.slice/crio-f28529540a1a7dc9af41f36dd6e9816f396e5d0f1af2595f91aeb330eb0d03e0 WatchSource:0}: Error finding container f28529540a1a7dc9af41f36dd6e9816f396e5d0f1af2595f91aeb330eb0d03e0: Status 404 returned error can't find the container with id f28529540a1a7dc9af41f36dd6e9816f396e5d0f1af2595f91aeb330eb0d03e0 Oct 30 18:42:06.335700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:06.335665 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" event={"ID":"77ae66cf-3dc0-451a-a5be-1f2b2e057465","Type":"ContainerStarted","Data":"64ada98826731da85a91c83f3e044b511a276f708f7a787f624be56a4e7b72d1"} Oct 30 18:42:06.336112 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:06.335726 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" event={"ID":"77ae66cf-3dc0-451a-a5be-1f2b2e057465","Type":"ContainerStarted","Data":"f28529540a1a7dc9af41f36dd6e9816f396e5d0f1af2595f91aeb330eb0d03e0"} Oct 30 18:42:06.336112 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:06.335860 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:06.379322 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:06.379249 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" podStartSLOduration=2.379225592 podStartE2EDuration="2.379225592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:06.375712151 +0000 UTC m=+596.913904359" watchObservedRunningTime="2026-10-30 18:42:06.379225592 +0000 UTC m=+596.917417797" Oct 30 18:42:08.236522 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:08.236305 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:42:08.236522 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:08.236369 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:42:09.328345 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:09.328313 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:42:09.977371 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:09.977269 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz"] Oct 30 18:42:09.982478 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:09.982450 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.002605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.002027 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-a759a1-serving-cert\"" Oct 30 18:42:10.003083 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.003049 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz"] Oct 30 18:42:10.024576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.024535 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.024756 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.024624 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k657z\" (UniqueName: \"kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.024756 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.024710 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.024872 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.024783 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.125979 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.125920 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.126137 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.126027 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.126199 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.126163 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.126199 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.126187 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k657z\" (UniqueName: \"kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.126348 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:42:10.126317 6160 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-serving-cert: secret "maas-api-e2e-worker-w1-a759a1-serving-cert" not found Oct 30 18:42:10.126477 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:42:10.126403 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls podName:6a4dc8b2-31c9-4776-bef0-61634e28f404 nodeName:}" failed. No retries permitted until 2026-10-30 18:42:10.626378771 +0000 UTC m=+601.164570973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls") pod "maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" (UID: "6a4dc8b2-31c9-4776-bef0-61634e28f404") : secret "maas-api-e2e-worker-w1-a759a1-serving-cert" not found Oct 30 18:42:10.126784 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.126763 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.128433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.128412 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.149348 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.149323 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k657z\" (UniqueName: \"kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.380621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.380579 6160 generic.go:356] "Generic (PLEG): container finished" podID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerID="a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093" exitCode=0 Oct 30 18:42:10.381230 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.380693 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" event={"ID":"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8","Type":"ContainerDied","Data":"a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093"} Oct 30 18:42:10.381304 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.381236 6160 scope.go:122] "RemoveContainer" containerID="a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093" Oct 30 18:42:10.382907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.382869 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:10.383038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.382979 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:10.383038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.383026 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:10.631198 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.631116 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.633496 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.633466 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:10.797977 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.791169 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:42:10.902038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:10.900891 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:11.249600 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:11.249104 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz"] Oct 30 18:42:11.406093 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:11.406043 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" event={"ID":"6a4dc8b2-31c9-4776-bef0-61634e28f404","Type":"ContainerStarted","Data":"c99c3a2d26f0c61096f294d76d6a9deedfdd1bb6c79e9698149da158dff4c1de"} Oct 30 18:42:11.408372 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:11.408344 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" event={"ID":"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8","Type":"ContainerStarted","Data":"9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698"} Oct 30 18:42:11.743089 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:11.743059 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:42:11.743198 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:11.743180 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:42:11.743262 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:11.743222 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Oct 30 18:42:12.439291 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:12.439207 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" event={"ID":"6a4dc8b2-31c9-4776-bef0-61634e28f404","Type":"ContainerStarted","Data":"df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880"} Oct 30 18:42:12.492980 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:12.492416 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" podStartSLOduration=3.054383536 podStartE2EDuration="3.492394249s" totalImagesPullingTime="438.010713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:12.491394773 +0000 UTC m=+603.029586992" watchObservedRunningTime="2026-10-30 18:42:12.492394249 +0000 UTC m=+603.030586476" Oct 30 18:42:12.744293 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:12.744205 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:42:12.744293 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:12.744254 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Oct 30 18:42:13.444214 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:13.444177 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:13.742657 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:13.742570 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:42:13.742657 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:13.742631 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Oct 30 18:42:14.428421 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:42:14.428351 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:42:14.743357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:14.743226 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:42:14.743357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:14.743295 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Oct 30 18:42:15.468008 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.462219 6160 generic.go:356] "Generic (PLEG): container finished" podID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerID="715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee" exitCode=0 Oct 30 18:42:15.468008 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.462368 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" event={"ID":"c5b0a55b-e180-49d3-a519-c79eabd7d6de","Type":"ContainerDied","Data":"715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee"} Oct 30 18:42:15.468008 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.462818 6160 scope.go:122] "RemoveContainer" containerID="715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee" Oct 30 18:42:15.468793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.468692 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:15.468793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.468787 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:15.469104 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.468839 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:15.743101 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.743006 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:42:15.743101 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:15.743055 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Oct 30 18:42:16.497010 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:16.495938 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" event={"ID":"c5b0a55b-e180-49d3-a519-c79eabd7d6de","Type":"ContainerStarted","Data":"80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28"} Oct 30 18:42:16.743004 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:16.742430 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:42:16.743004 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:16.742494 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Oct 30 18:42:17.343469 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.343441 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:42:17.492003 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.491329 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:42:17.492366 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.492337 6160 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:42:17.492431 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.492395 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Oct 30 18:42:17.746336 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.746305 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:42:17.746782 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.746758 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:42:17.747469 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:17.747450 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:42:18.484382 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:18.484351 6160 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:42:18.484641 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:18.484611 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Oct 30 18:42:19.479424 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.479327 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:42:19.484076 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.484043 6160 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:42:19.484231 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.484102 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Oct 30 18:42:19.524468 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.524436 6160 generic.go:356] "Generic (PLEG): container finished" podID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerID="fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd" exitCode=0 Oct 30 18:42:19.524609 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.524500 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" event={"ID":"fb1cb278-d548-45cd-8919-cc38e0be4554","Type":"ContainerDied","Data":"fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd"} Oct 30 18:42:19.524981 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.524968 6160 scope.go:122] "RemoveContainer" containerID="fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd" Oct 30 18:42:19.526328 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.526295 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:19.526446 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.526400 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:19.526446 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:19.526436 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:20.193037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.192146 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:42:20.487907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.487391 6160 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:42:20.487907 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.487464 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Oct 30 18:42:20.536968 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.531615 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" event={"ID":"fb1cb278-d548-45cd-8919-cc38e0be4554","Type":"ContainerStarted","Data":"5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252"} Oct 30 18:42:20.960975 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.960315 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:42:20.980978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.971157 6160 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:42:20.980978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:20.971254 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Oct 30 18:42:21.506037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:21.496043 6160 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:42:21.506037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:21.496104 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Oct 30 18:42:21.962369 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:21.962330 6160 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:42:21.962648 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:21.962617 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Oct 30 18:42:22.194133 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:22.194086 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:42:22.490404 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:22.490370 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:42:22.548644 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:22.548603 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:42:22.962920 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:22.962886 6160 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:42:22.963212 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:22.963184 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Oct 30 18:42:23.198524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.198492 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:42:23.569026 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.568990 6160 generic.go:356] "Generic (PLEG): container finished" podID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerID="a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d" exitCode=0 Oct 30 18:42:23.570483 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.570456 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" event={"ID":"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96","Type":"ContainerDied","Data":"a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d"} Oct 30 18:42:23.578007 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.572938 6160 scope.go:122] "RemoveContainer" containerID="a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d" Oct 30 18:42:23.579294 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.579260 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:23.579398 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.579346 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:23.579465 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.579394 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:42:23.958804 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.958708 6160 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:42:23.959074 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:23.959043 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Oct 30 18:42:24.204403 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:24.204362 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:42:24.603309 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:24.603268 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" event={"ID":"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96","Type":"ContainerStarted","Data":"98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f"} Oct 30 18:42:24.963025 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:24.959325 6160 patch_prober.go:29] interesting pod/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:42:24.963025 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:24.959390 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Oct 30 18:42:25.519202 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:25.519166 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:42:25.520234 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:25.520209 6160 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:42:25.520289 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:25.520255 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Oct 30 18:42:25.980896 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:25.980866 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:42:25.981816 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:25.981798 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:42:26.519451 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:26.519417 6160 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:42:26.519607 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:26.519465 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Oct 30 18:42:27.518797 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:27.518762 6160 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:42:27.519240 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:27.518831 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Oct 30 18:42:28.519410 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:28.519372 6160 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:42:28.519826 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:28.519426 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Oct 30 18:42:29.518940 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:29.518902 6160 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:42:29.519132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:29.518994 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Oct 30 18:42:30.523423 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:30.523395 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:42:30.523822 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:30.523673 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:42:30.524445 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:30.524423 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:42:31.295152 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:31.295120 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:42:42.072868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:42.072829 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:42:44.395452 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:42:44.395414 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:42:54.486836 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.486796 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm"] Oct 30 18:42:54.490612 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.490591 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.515446 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.515415 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"kube-root-ca.crt\"" Oct 30 18:42:54.524509 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.524476 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"default-dockercfg-szghr\"" Oct 30 18:42:54.527244 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.527224 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"premium-w2-b2ef42-kserve-self-signed-certs\"" Oct 30 18:42:54.529012 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.528990 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm"] Oct 30 18:42:54.583912 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.583882 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.584069 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.583925 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.584113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.584091 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.584156 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.584116 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.584156 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.584148 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.584260 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.584246 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4426z\" (UniqueName: \"kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.613066 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.613042 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-b2ef42\"/\"openshift-service-ca.crt\"" Oct 30 18:42:54.685742 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.685709 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4426z\" (UniqueName: \"kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.686113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.685790 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.686113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.685828 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.686113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.685971 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.686113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.686000 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.686113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.686057 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.687129 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.687102 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.687735 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.687705 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.689429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.689392 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.701999 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.692085 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.701999 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.701902 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.717928 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.717763 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4426z\" (UniqueName: \"kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z\") pod \"premium-w2-b2ef42-kserve-7c5b698c47-gtqkm\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:54.814322 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:54.814288 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:55.128436 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:55.128350 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm"] Oct 30 18:42:55.132254 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:42:55.132224 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdeebc55_1539_473e_a93a_b19851f0cf90.slice/crio-6055d2af39f41eede45e6399e506a6eb42140cb18f9abbf537ffa32ffe5554ac WatchSource:0}: Error finding container 6055d2af39f41eede45e6399e506a6eb42140cb18f9abbf537ffa32ffe5554ac: Status 404 returned error can't find the container with id 6055d2af39f41eede45e6399e506a6eb42140cb18f9abbf537ffa32ffe5554ac Oct 30 18:42:55.766085 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:55.766049 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" event={"ID":"bdeebc55-1539-473e-a93a-b19851f0cf90","Type":"ContainerStarted","Data":"17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33"} Oct 30 18:42:55.766472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:55.766097 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" event={"ID":"bdeebc55-1539-473e-a93a-b19851f0cf90","Type":"ContainerStarted","Data":"6055d2af39f41eede45e6399e506a6eb42140cb18f9abbf537ffa32ffe5554ac"} Oct 30 18:42:55.808105 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:55.807923 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" podStartSLOduration=1.807904405 podStartE2EDuration="1.807904405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:55.806582323 +0000 UTC m=+646.344774528" watchObservedRunningTime="2026-10-30 18:42:55.807904405 +0000 UTC m=+646.346096612" Oct 30 18:42:55.815353 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:55.815325 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:42:57.486581 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:42:57.486549 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:43:04.814477 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:04.814446 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:43:04.827480 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:04.827457 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:43:05.824989 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:05.824930 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:43:07.319702 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.319656 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2"] Oct 30 18:43:07.322481 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.322458 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.341984 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.341944 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"openshift-service-ca.crt\"" Oct 30 18:43:07.354059 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.354035 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"premium-w3-0ddc26-kserve-self-signed-certs\"" Oct 30 18:43:07.354199 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.354045 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"kube-root-ca.crt\"" Oct 30 18:43:07.354322 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.354262 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"default-dockercfg-vnsmm\"" Oct 30 18:43:07.355850 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.355831 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2"] Oct 30 18:43:07.419921 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.419895 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.420085 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.419975 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.420132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.420082 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.420174 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.420158 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z6hx\" (UniqueName: \"kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.420210 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.420200 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.420253 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.420235 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.532207 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532156 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.532412 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532309 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.532412 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532350 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.532530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532414 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z6hx\" (UniqueName: \"kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.532530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532474 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.532530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532503 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.534715 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.532970 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.536455 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.536156 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.536455 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.536391 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.542269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.540453 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.542269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.541071 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.559905 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.559867 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z6hx\" (UniqueName: \"kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx\") pod \"premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.635164 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.635100 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:07.929355 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:07.929276 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2"] Oct 30 18:43:07.933164 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:43:07.933140 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e702456_b696_41e1_b638_b9bb1a344f23.slice/crio-276e5217e500bfd52c7bb7226df831d2a061f3f6b28581c77961e84c4c94e396 WatchSource:0}: Error finding container 276e5217e500bfd52c7bb7226df831d2a061f3f6b28581c77961e84c4c94e396: Status 404 returned error can't find the container with id 276e5217e500bfd52c7bb7226df831d2a061f3f6b28581c77961e84c4c94e396 Oct 30 18:43:08.830698 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:08.830662 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" event={"ID":"6e702456-b696-41e1-b638-b9bb1a344f23","Type":"ContainerStarted","Data":"21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac"} Oct 30 18:43:08.831109 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:08.830707 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" event={"ID":"6e702456-b696-41e1-b638-b9bb1a344f23","Type":"ContainerStarted","Data":"276e5217e500bfd52c7bb7226df831d2a061f3f6b28581c77961e84c4c94e396"} Oct 30 18:43:08.873282 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:08.873235 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" podStartSLOduration=1.8732174110000002 podStartE2EDuration="1.873217411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:08.871488503 +0000 UTC m=+659.409680706" watchObservedRunningTime="2026-10-30 18:43:08.873217411 +0000 UTC m=+659.411409617" Oct 30 18:43:09.635674 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:09.635637 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:13.140780 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.140746 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft"] Oct 30 18:43:13.143274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.143250 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.162144 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.162123 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"openshift-service-ca.crt\"" Oct 30 18:43:13.162144 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.162140 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"kube-root-ca.crt\"" Oct 30 18:43:13.173974 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.173936 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"default-dockercfg-5q6fx\"" Oct 30 18:43:13.176072 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.176051 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft"] Oct 30 18:43:13.181703 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.181678 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.181792 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.181732 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8sdh\" (UniqueName: \"kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.181851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.181829 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.181962 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.181935 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.182019 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.182005 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.182055 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.182025 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.263535 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.263500 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f6f7d5\"/\"premium-w0-f6f7d5-kserve-self-signed-certs\"" Oct 30 18:43:13.283017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.282979 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283072 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283120 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283142 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283310 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283277 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283358 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283317 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8sdh\" (UniqueName: \"kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283509 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283483 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283618 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283600 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.283688 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.283669 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.285316 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.285294 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.285488 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.285472 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.320917 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.320848 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8sdh\" (UniqueName: \"kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh\") pod \"premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.453943 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.453853 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:13.747749 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.747665 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft"] Oct 30 18:43:13.751628 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:43:13.751596 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2857cfa_12ba_424e_90bd_4f481b977654.slice/crio-e12b18fe84a51c3ef076fc09bb7b1d6faad848beca6e1625a3ea0fa351c7010a WatchSource:0}: Error finding container e12b18fe84a51c3ef076fc09bb7b1d6faad848beca6e1625a3ea0fa351c7010a: Status 404 returned error can't find the container with id e12b18fe84a51c3ef076fc09bb7b1d6faad848beca6e1625a3ea0fa351c7010a Oct 30 18:43:13.858891 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.858854 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" event={"ID":"f2857cfa-12ba-424e-90bd-4f481b977654","Type":"ContainerStarted","Data":"8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a"} Oct 30 18:43:13.859096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.858902 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" event={"ID":"f2857cfa-12ba-424e-90bd-4f481b977654","Type":"ContainerStarted","Data":"e12b18fe84a51c3ef076fc09bb7b1d6faad848beca6e1625a3ea0fa351c7010a"} Oct 30 18:43:13.899965 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:13.899900 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" podStartSLOduration=0.899882833 podStartE2EDuration="899.882833ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:13.898890464 +0000 UTC m=+664.437082669" watchObservedRunningTime="2026-10-30 18:43:13.899882833 +0000 UTC m=+664.438075038" Oct 30 18:43:14.396302 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:43:14.396255 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:43:14.454059 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:14.454034 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:15.191320 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:15.191281 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:43:17.635642 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:17.635607 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:17.648424 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:17.648393 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:17.891261 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:17.891183 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:43:23.454427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:23.454394 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:23.467136 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:23.467110 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:23.920155 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:23.920123 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:43:38.166534 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:43:38.166499 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:43:44.395428 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:43:44.395387 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:44:14.395939 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:14.395892 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:44:17.025800 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.025759 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9"] Oct 30 18:44:17.028567 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.028547 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.043184 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.043165 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-jbnzt\"" Oct 30 18:44:17.055258 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.055228 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-61485f235dab\"" Oct 30 18:44:17.055464 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.055228 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\"" Oct 30 18:44:17.055464 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.055253 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\"" Oct 30 18:44:17.055643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.055243 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\"" Oct 30 18:44:17.055643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.055346 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\"" Oct 30 18:44:17.058223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.058196 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9"] Oct 30 18:44:17.128046 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.128003 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shf5g\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.128242 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.128068 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.128242 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.128124 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.128354 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.128269 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.143189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.143160 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-93f8ac45085c\"" Oct 30 18:44:17.229245 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.229194 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shf5g\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.229245 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.229237 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.229514 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.229270 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.229514 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.229371 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.230033 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.229998 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.230248 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.230227 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.231970 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.231938 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.252124 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.252098 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shf5g\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g\") pod \"payload-processing-external-model-77cdcd47c-pbnr9\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.340790 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.340759 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:17.641374 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:17.641296 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9"] Oct 30 18:44:17.645492 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:44:17.645459 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658061a5_1a2d_4798_a7ef_ac23529ddba3.slice/crio-418969f41a9516c9434aaa79555abfaade05a2558aff683389eb69362cef9abb WatchSource:0}: Error finding container 418969f41a9516c9434aaa79555abfaade05a2558aff683389eb69362cef9abb: Status 404 returned error can't find the container with id 418969f41a9516c9434aaa79555abfaade05a2558aff683389eb69362cef9abb Oct 30 18:44:18.223492 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:18.223443 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" event={"ID":"658061a5-1a2d-4798-a7ef-ac23529ddba3","Type":"ContainerStarted","Data":"48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52"} Oct 30 18:44:18.224029 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:18.223533 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" event={"ID":"658061a5-1a2d-4798-a7ef-ac23529ddba3","Type":"ContainerStarted","Data":"418969f41a9516c9434aaa79555abfaade05a2558aff683389eb69362cef9abb"} Oct 30 18:44:18.224555 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:18.224510 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:27.895236 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:27.895095 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" podStartSLOduration=10.895079424 podStartE2EDuration="10.895079424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:18.269168439 +0000 UTC m=+728.807360645" watchObservedRunningTime="2026-10-30 18:44:27.895079424 +0000 UTC m=+738.433271631" Oct 30 18:44:27.897125 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:27.897097 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:44:30.238739 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:30.238710 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:44:38.777302 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.777270 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg"] Oct 30 18:44:38.780796 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.780780 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.817785 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.817753 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg"] Oct 30 18:44:38.826614 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.826583 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg"] Oct 30 18:44:38.830343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.830324 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:38.896665 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.896556 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897036 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897004 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897151 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897277 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897434 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897556 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897532 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrwm6\" (UniqueName: \"kubernetes.io/projected/54ce7764-e6ec-45fd-b490-51c7c5f531f5-kube-api-access-lrwm6\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897660 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897646 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.897986 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.897943 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.898282 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.898265 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.955117 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.954437 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg"] Oct 30 18:44:38.999306 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999267 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.999472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999314 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrwm6\" (UniqueName: \"kubernetes.io/projected/54ce7764-e6ec-45fd-b490-51c7c5f531f5-kube-api-access-lrwm6\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.999472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999435 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.999589 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999507 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf6fh\" (UniqueName: \"kubernetes.io/projected/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-kube-api-access-nf6fh\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:38.999589 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999560 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:38.999689 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999638 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:38.999689 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999669 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.999689 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999678 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.999850 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999697 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:38.999850 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999778 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:38.999850 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999815 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.000103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999859 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:38.999930 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000008 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000029 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000078 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000101 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.000416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000122 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.000416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000160 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.000416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000195 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.000575 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000424 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000575 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000520 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.000687 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.000571 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.002047 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.002022 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.002381 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.002361 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.034968 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.034938 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54ce7764-e6ec-45fd-b490-51c7c5f531f5-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.043243 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.043222 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrwm6\" (UniqueName: \"kubernetes.io/projected/54ce7764-e6ec-45fd-b490-51c7c5f531f5-kube-api-access-lrwm6\") pod \"maas-default-gateway-openshift-default-6dcf757899-qnfsg\" (UID: \"54ce7764-e6ec-45fd-b490-51c7c5f531f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.096920 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.096895 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.100826 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf6fh\" (UniqueName: \"kubernetes.io/projected/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-kube-api-access-nf6fh\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.100872 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.100892 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.100922 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.100941 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101012 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101028 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101053 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101082 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101265 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101776 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101555 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101776 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101605 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101776 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101651 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.101776 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.101688 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.103420 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.103397 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.103651 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.103628 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.125462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.125434 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.125583 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.125569 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf6fh\" (UniqueName: \"kubernetes.io/projected/2c4b38b0-ea36-4f64-9e8e-927edf4a45e8-kube-api-access-nf6fh\") pod \"maas-default-gateway-openshift-default-6dcf757899-tqkjg\" (UID: \"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.144996 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.144948 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:39.299655 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.299583 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp"] Oct 30 18:44:39.304177 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.304158 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.334239 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.334216 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp"] Oct 30 18:44:39.404187 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.404156 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb5vk\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.404361 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.404221 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.404361 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.404334 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.404445 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.404373 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.418676 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.418645 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg"] Oct 30 18:44:39.422484 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:44:39.422456 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ce7764_e6ec_45fd_b490_51c7c5f531f5.slice/crio-d906942a0a7ddb562e94af7679806a0e127409d257850442945ad866b49a1c87 WatchSource:0}: Error finding container d906942a0a7ddb562e94af7679806a0e127409d257850442945ad866b49a1c87: Status 404 returned error can't find the container with id d906942a0a7ddb562e94af7679806a0e127409d257850442945ad866b49a1c87 Oct 30 18:44:39.424750 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.424720 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:44:39.424833 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.424791 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:44:39.424833 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.424821 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:44:39.476946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.476922 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg"] Oct 30 18:44:39.480525 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:44:39.480499 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4b38b0_ea36_4f64_9e8e_927edf4a45e8.slice/crio-f676edb5dbf1a607a63c9553089e40ad7af4f6568f8df06ad1ccf4812b95771b WatchSource:0}: Error finding container f676edb5dbf1a607a63c9553089e40ad7af4f6568f8df06ad1ccf4812b95771b: Status 404 returned error can't find the container with id f676edb5dbf1a607a63c9553089e40ad7af4f6568f8df06ad1ccf4812b95771b Oct 30 18:44:39.483209 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.483171 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:44:39.483277 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.483253 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:44:39.483325 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.483301 6160 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Oct 30 18:44:39.505159 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.505134 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb5vk\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.505274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.505183 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.505274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.505211 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.505274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.505237 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.505868 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.505849 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.506129 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.506103 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.507441 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.507422 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.530307 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.529364 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb5vk\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk\") pod \"payload-processing-external-model-58db6964f9-4h2sp\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.617922 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.617842 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:39.947470 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:39.947378 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp"] Oct 30 18:44:39.952898 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:44:39.952863 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1c88c6_6a78_425b_b47a_91194fdae902.slice/crio-bbda70021607fe008c2c815c71602dd7e4001b48fb34292b3d3158737e1e3a64 WatchSource:0}: Error finding container bbda70021607fe008c2c815c71602dd7e4001b48fb34292b3d3158737e1e3a64: Status 404 returned error can't find the container with id bbda70021607fe008c2c815c71602dd7e4001b48fb34292b3d3158737e1e3a64 Oct 30 18:44:40.386507 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.386460 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" event={"ID":"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8","Type":"ContainerStarted","Data":"1e00d88f80d108f69096ec7ee60f8256c7a02fa566082bc98a51acc8e748f63f"} Oct 30 18:44:40.386793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.386755 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" event={"ID":"2c4b38b0-ea36-4f64-9e8e-927edf4a45e8","Type":"ContainerStarted","Data":"f676edb5dbf1a607a63c9553089e40ad7af4f6568f8df06ad1ccf4812b95771b"} Oct 30 18:44:40.390461 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.390428 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" event={"ID":"54ce7764-e6ec-45fd-b490-51c7c5f531f5","Type":"ContainerStarted","Data":"b80dd1b583b7da6311437a12da6527b1e3c7b3f07d5532a693ff22306b3390f4"} Oct 30 18:44:40.390656 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.390637 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" event={"ID":"54ce7764-e6ec-45fd-b490-51c7c5f531f5","Type":"ContainerStarted","Data":"d906942a0a7ddb562e94af7679806a0e127409d257850442945ad866b49a1c87"} Oct 30 18:44:40.394278 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.394251 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" event={"ID":"7d1c88c6-6a78-425b-b47a-91194fdae902","Type":"ContainerStarted","Data":"c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3"} Oct 30 18:44:40.394470 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.394452 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" event={"ID":"7d1c88c6-6a78-425b-b47a-91194fdae902","Type":"ContainerStarted","Data":"bbda70021607fe008c2c815c71602dd7e4001b48fb34292b3d3158737e1e3a64"} Oct 30 18:44:40.395434 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.395417 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:40.439618 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.439566 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podStartSLOduration=2.4395505760000002 podStartE2EDuration="2.439550576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.437547705 +0000 UTC m=+750.975739913" watchObservedRunningTime="2026-10-30 18:44:40.439550576 +0000 UTC m=+750.977742782" Oct 30 18:44:40.492682 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.492611 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" podStartSLOduration=1.492592318 podStartE2EDuration="1.492592318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.49163891 +0000 UTC m=+751.029831116" watchObservedRunningTime="2026-10-30 18:44:40.492592318 +0000 UTC m=+751.030784523" Oct 30 18:44:40.535331 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:40.535299 6160 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-61485f235dab: secret "agc-cred-61485f235dab" not found Oct 30 18:44:40.535331 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:40.535327 6160 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp: secret "agc-cred-61485f235dab" not found Oct 30 18:44:40.535518 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:40.535383 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials podName:7d1c88c6-6a78-425b-b47a-91194fdae902 nodeName:}" failed. No retries permitted until 2026-10-30 18:44:41.035369086 +0000 UTC m=+751.573561284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials") pod "payload-processing-external-model-58db6964f9-4h2sp" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902") : secret "agc-cred-61485f235dab" not found Oct 30 18:44:40.539033 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:40.537888 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podStartSLOduration=2.537874866 podStartE2EDuration="2.537874866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.536862846 +0000 UTC m=+751.075055052" watchObservedRunningTime="2026-10-30 18:44:40.537874866 +0000 UTC m=+751.076067071" Oct 30 18:44:41.038854 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:41.038822 6160 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-61485f235dab: secret "agc-cred-61485f235dab" not found Oct 30 18:44:41.038854 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:41.038847 6160 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp: secret "agc-cred-61485f235dab" not found Oct 30 18:44:41.039309 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:41.038910 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials podName:7d1c88c6-6a78-425b-b47a-91194fdae902 nodeName:}" failed. No retries permitted until 2026-10-30 18:44:42.038894722 +0000 UTC m=+752.577086906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials") pod "payload-processing-external-model-58db6964f9-4h2sp" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902") : secret "agc-cred-61485f235dab" not found Oct 30 18:44:41.102131 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.102043 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:41.103891 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.103577 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.103891 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.103635 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:41.146023 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.145994 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:41.147120 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.147099 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.147195 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.147146 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:41.278594 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:41.278556 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp"] Oct 30 18:44:42.049961 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:42.049918 6160 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-61485f235dab: secret "agc-cred-61485f235dab" not found Oct 30 18:44:42.049961 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:42.049945 6160 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp: secret "agc-cred-61485f235dab" not found Oct 30 18:44:42.050389 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:42.050018 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials podName:7d1c88c6-6a78-425b-b47a-91194fdae902 nodeName:}" failed. No retries permitted until 2026-10-30 18:44:44.050003954 +0000 UTC m=+754.588196157 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials") pod "payload-processing-external-model-58db6964f9-4h2sp" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902") : secret "agc-cred-61485f235dab" not found Oct 30 18:44:42.103021 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:42.101399 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.103021 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:42.101459 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:42.146308 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:42.146267 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.146481 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:42.146330 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:43.099415 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:43.099373 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.100035 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:43.100002 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:43.146287 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:43.146253 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.146471 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:43.146308 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:44.097977 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:44.097427 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:44.097977 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:44.097497 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:44.114817 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:44.114781 6160 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-61485f235dab: secret "agc-cred-61485f235dab" not found Oct 30 18:44:44.114817 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:44.114811 6160 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp: secret "agc-cred-61485f235dab" not found Oct 30 18:44:44.115268 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:44.114860 6160 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials podName:7d1c88c6-6a78-425b-b47a-91194fdae902 nodeName:}" failed. No retries permitted until 2026-10-30 18:44:48.114844496 +0000 UTC m=+758.653036692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials") pod "payload-processing-external-model-58db6964f9-4h2sp" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902") : secret "agc-cred-61485f235dab" not found Oct 30 18:44:44.146145 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:44.146111 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:44.146316 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:44.146163 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:44.440586 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:44.440487 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:44:45.101573 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:45.101400 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:45.101573 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:45.101477 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:45.145901 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:45.145871 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:45.146319 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:45.145919 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:45.215344 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:45.215285 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:44:46.098085 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:46.098041 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:46.098311 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:46.098099 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:46.146221 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:46.146182 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:46.146741 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:46.146252 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:47.097645 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.097603 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:44:47.097830 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.097668 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Oct 30 18:44:47.145541 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.145512 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:44:47.145734 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.145557 6160 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Oct 30 18:44:47.419215 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.419093 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" podUID="7d1c88c6-6a78-425b-b47a-91194fdae902" containerName="payload-processing" containerID="cri-o://c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3" gracePeriod=25 Oct 30 18:44:47.877375 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.877351 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.971011 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk\") pod \"7d1c88c6-6a78-425b-b47a-91194fdae902\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.971510 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume\") pod \"7d1c88c6-6a78-425b-b47a-91194fdae902\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.971937 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume" pod "7d1c88c6-6a78-425b-b47a-91194fdae902" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.972056 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials\") pod \"7d1c88c6-6a78-425b-b47a-91194fdae902\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.972528 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume\") pod \"7d1c88c6-6a78-425b-b47a-91194fdae902\" (UID: \"7d1c88c6-6a78-425b-b47a-91194fdae902\") " Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.972991 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume" pod "7d1c88c6-6a78-425b-b47a-91194fdae902" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.973387 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:44:47.974416 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.973408 6160 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7d1c88c6-6a78-425b-b47a-91194fdae902-routing-overlay-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:44:47.982070 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.977354 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk" pod "7d1c88c6-6a78-425b-b47a-91194fdae902" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902"). InnerVolumeSpecName "kube-api-access-nb5vk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:47.990330 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:47.990229 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials" pod "7d1c88c6-6a78-425b-b47a-91194fdae902" (UID: "7d1c88c6-6a78-425b-b47a-91194fdae902"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:48.074553 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.074513 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb5vk\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-kube-api-access-nb5vk\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:44:48.075130 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.075105 6160 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7d1c88c6-6a78-425b-b47a-91194fdae902-provider-credentials\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:44:48.473133 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.473099 6160 generic.go:356] "Generic (PLEG): container finished" podID="7d1c88c6-6a78-425b-b47a-91194fdae902" containerID="c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3" exitCode=0 Oct 30 18:44:48.473615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.473158 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" Oct 30 18:44:48.473615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.473193 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" event={"ID":"7d1c88c6-6a78-425b-b47a-91194fdae902","Type":"ContainerDied","Data":"c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3"} Oct 30 18:44:48.473615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.473246 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp" event={"ID":"7d1c88c6-6a78-425b-b47a-91194fdae902","Type":"ContainerDied","Data":"bbda70021607fe008c2c815c71602dd7e4001b48fb34292b3d3158737e1e3a64"} Oct 30 18:44:48.473615 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.473277 6160 scope.go:122] "RemoveContainer" containerID="c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3" Oct 30 18:44:48.484506 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.484488 6160 scope.go:122] "RemoveContainer" containerID="c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3" Oct 30 18:44:48.484791 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:44:48.484769 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3\": container with ID starting with c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3 not found: ID does not exist" containerID="c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3" Oct 30 18:44:48.484856 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.484798 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3"} err="failed to get container status \"c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3\": rpc error: code = NotFound desc = could not find container \"c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3\": container with ID starting with c7429a4309626607c5b3be6eeaf0b3d53e6783a083486e2015f43d67a3fb20f3 not found: ID does not exist" Oct 30 18:44:48.528017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.527981 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp"] Oct 30 18:44:48.538587 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.538554 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-58db6964f9-4h2sp"] Oct 30 18:44:48.574311 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.574277 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:48.575794 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.575669 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:48.577208 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.577181 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" Oct 30 18:44:48.642700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.642667 6160 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:48.643630 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.643596 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:48.644659 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:48.644527 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" Oct 30 18:44:49.582129 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:49.582090 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d1c88c6-6a78-425b-b47a-91194fdae902" path="/var/lib/kubelet/pods/7d1c88c6-6a78-425b-b47a-91194fdae902/volumes" Oct 30 18:44:54.621585 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:54.621550 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9"] Oct 30 18:44:59.621967 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:59.621922 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" podUID="658061a5-1a2d-4798-a7ef-ac23529ddba3" containerName="payload-processing" containerID="cri-o://48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52" gracePeriod=25 Oct 30 18:44:59.925756 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:44:59.925733 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:45:00.037443 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.037411 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g\") pod \"658061a5-1a2d-4798-a7ef-ac23529ddba3\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " Oct 30 18:45:00.037621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.037456 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume\") pod \"658061a5-1a2d-4798-a7ef-ac23529ddba3\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " Oct 30 18:45:00.037621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.037483 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume\") pod \"658061a5-1a2d-4798-a7ef-ac23529ddba3\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " Oct 30 18:45:00.037621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.037545 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials\") pod \"658061a5-1a2d-4798-a7ef-ac23529ddba3\" (UID: \"658061a5-1a2d-4798-a7ef-ac23529ddba3\") " Oct 30 18:45:00.037815 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.037785 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume" pod "658061a5-1a2d-4798-a7ef-ac23529ddba3" (UID: "658061a5-1a2d-4798-a7ef-ac23529ddba3"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:00.037976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.037929 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume" pod "658061a5-1a2d-4798-a7ef-ac23529ddba3" (UID: "658061a5-1a2d-4798-a7ef-ac23529ddba3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:00.038079 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.038064 6160 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-routing-overlay-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:00.038126 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.038081 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/658061a5-1a2d-4798-a7ef-ac23529ddba3-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:00.039726 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.039687 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials" pod "658061a5-1a2d-4798-a7ef-ac23529ddba3" (UID: "658061a5-1a2d-4798-a7ef-ac23529ddba3"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:00.039825 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.039761 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g" pod "658061a5-1a2d-4798-a7ef-ac23529ddba3" (UID: "658061a5-1a2d-4798-a7ef-ac23529ddba3"). InnerVolumeSpecName "kube-api-access-shf5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:00.138774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.138743 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shf5g\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-kube-api-access-shf5g\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:00.138774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.138769 6160 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/658061a5-1a2d-4798-a7ef-ac23529ddba3-provider-credentials\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:00.542646 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.542613 6160 generic.go:356] "Generic (PLEG): container finished" podID="658061a5-1a2d-4798-a7ef-ac23529ddba3" containerID="48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52" exitCode=0 Oct 30 18:45:00.542848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.542673 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" Oct 30 18:45:00.542848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.542701 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" event={"ID":"658061a5-1a2d-4798-a7ef-ac23529ddba3","Type":"ContainerDied","Data":"48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52"} Oct 30 18:45:00.542848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.542752 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9" event={"ID":"658061a5-1a2d-4798-a7ef-ac23529ddba3","Type":"ContainerDied","Data":"418969f41a9516c9434aaa79555abfaade05a2558aff683389eb69362cef9abb"} Oct 30 18:45:00.542848 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.542775 6160 scope.go:122] "RemoveContainer" containerID="48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52" Oct 30 18:45:00.552815 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.552800 6160 scope.go:122] "RemoveContainer" containerID="48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52" Oct 30 18:45:00.553158 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:45:00.553136 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52\": container with ID starting with 48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52 not found: ID does not exist" containerID="48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52" Oct 30 18:45:00.553240 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.553166 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52"} err="failed to get container status \"48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52\": rpc error: code = NotFound desc = could not find container \"48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52\": container with ID starting with 48e58ec87a2b16a63180483c525fc931ee28c8316c675b08d8423a7c5f25cb52 not found: ID does not exist" Oct 30 18:45:00.595588 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.595442 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9"] Oct 30 18:45:00.606846 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:00.606816 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-77cdcd47c-pbnr9"] Oct 30 18:45:01.578386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:01.578355 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="658061a5-1a2d-4798-a7ef-ac23529ddba3" path="/var/lib/kubelet/pods/658061a5-1a2d-4798-a7ef-ac23529ddba3/volumes" Oct 30 18:45:02.033866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.033831 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs"] Oct 30 18:45:02.034348 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.034332 6160 cpu_manager.go:414] "containerMap: removing container" podUID="7d1c88c6-6a78-425b-b47a-91194fdae902" containerName="payload-processing" Oct 30 18:45:02.034348 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.034349 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d1c88c6-6a78-425b-b47a-91194fdae902" containerName="payload-processing" Oct 30 18:45:02.034453 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.034375 6160 cpu_manager.go:414] "containerMap: removing container" podUID="658061a5-1a2d-4798-a7ef-ac23529ddba3" containerName="payload-processing" Oct 30 18:45:02.034453 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.034381 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="658061a5-1a2d-4798-a7ef-ac23529ddba3" containerName="payload-processing" Oct 30 18:45:02.034511 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.034456 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d1c88c6-6a78-425b-b47a-91194fdae902" containerName="payload-processing" Oct 30 18:45:02.034511 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.034467 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="658061a5-1a2d-4798-a7ef-ac23529ddba3" containerName="payload-processing" Oct 30 18:45:02.038864 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.038847 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.055156 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.055130 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\"" Oct 30 18:45:02.055278 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.055177 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\"" Oct 30 18:45:02.055278 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.055221 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\"" Oct 30 18:45:02.055278 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.055248 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\"" Oct 30 18:45:02.067329 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.067311 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-c2afa8fb9814\"" Oct 30 18:45:02.067450 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.067334 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-7779a3579b77\"" Oct 30 18:45:02.067536 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.067361 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-jbnzt\"" Oct 30 18:45:02.067536 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.067370 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-93f8ac45085c\"" Oct 30 18:45:02.069707 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.069686 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs"] Oct 30 18:45:02.179077 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.179039 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd4dh\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.179233 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.179187 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.179327 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.179307 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.179390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.179340 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.280243 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.280208 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.280427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.280279 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.280427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.280300 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.280427 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.280393 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd4dh\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.281130 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.281095 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.281255 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.281215 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.282922 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.282904 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.303440 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.303383 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd4dh\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh\") pod \"payload-processing-external-model-5d7cd86ff8-98ffs\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.351812 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.351792 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:02.647579 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:02.647499 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs"] Oct 30 18:45:02.651202 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:45:02.651175 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a7c557_fb81_4d7b_8ce3_0b7c5e663439.slice/crio-6f55c44d1d9dc916c8006a2ab63600098ea4b9f41226f0a234eab7f7317c4bf0 WatchSource:0}: Error finding container 6f55c44d1d9dc916c8006a2ab63600098ea4b9f41226f0a234eab7f7317c4bf0: Status 404 returned error can't find the container with id 6f55c44d1d9dc916c8006a2ab63600098ea4b9f41226f0a234eab7f7317c4bf0 Oct 30 18:45:03.583064 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:03.583035 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:03.583235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:03.583085 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" event={"ID":"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439","Type":"ContainerStarted","Data":"1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39"} Oct 30 18:45:03.583235 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:03.583108 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" event={"ID":"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439","Type":"ContainerStarted","Data":"6f55c44d1d9dc916c8006a2ab63600098ea4b9f41226f0a234eab7f7317c4bf0"} Oct 30 18:45:03.619573 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:03.619515 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" podStartSLOduration=1.619501454 podStartE2EDuration="1.619501454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:03.617514061 +0000 UTC m=+774.155706266" watchObservedRunningTime="2026-10-30 18:45:03.619501454 +0000 UTC m=+774.157693656" Oct 30 18:45:07.903183 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:07.903149 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:45:14.396285 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:45:14.396245 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:45:14.583322 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:14.583294 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:19.638097 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:19.638055 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qnfsg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:45:19.638861 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:19.638823 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qnfsg" podUID="54ce7764-e6ec-45fd-b490-51c7c5f531f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:45:19.690372 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:19.689881 6160 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tqkjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:45:19.690372 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:19.689994 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tqkjg" podUID="2c4b38b0-ea36-4f64-9e8e-927edf4a45e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:45:27.008256 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:27.008222 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:45:34.588496 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.588463 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-z44ct"] Oct 30 18:45:34.592459 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.592437 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.612840 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.612810 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-z44ct"] Oct 30 18:45:34.716663 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.716622 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.716878 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.716706 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62lw4\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.716878 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.716834 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.717064 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.716904 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.818353 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.818312 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.818530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.818414 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.818530 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.818460 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62lw4\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.818644 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.818534 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.819290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.819255 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.819419 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.819261 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.821318 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.821288 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.858789 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.858733 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62lw4\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4\") pod \"payload-processing-external-model-d66597b7c-z44ct\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:34.912449 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:34.911160 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:35.247639 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:35.247565 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-z44ct"] Oct 30 18:45:35.251177 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:45:35.251151 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8829315_056f_40bc_8c72_aca7c9e62569.slice/crio-46e64622409d3ab7892230f523837dcbb9c65fb6f8a64fc9b9483ea20d2b5079 WatchSource:0}: Error finding container 46e64622409d3ab7892230f523837dcbb9c65fb6f8a64fc9b9483ea20d2b5079: Status 404 returned error can't find the container with id 46e64622409d3ab7892230f523837dcbb9c65fb6f8a64fc9b9483ea20d2b5079 Oct 30 18:45:35.799968 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:35.799930 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" event={"ID":"c8829315-056f-40bc-8c72-aca7c9e62569","Type":"ContainerStarted","Data":"fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954"} Oct 30 18:45:35.800338 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:35.800004 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" event={"ID":"c8829315-056f-40bc-8c72-aca7c9e62569","Type":"ContainerStarted","Data":"46e64622409d3ab7892230f523837dcbb9c65fb6f8a64fc9b9483ea20d2b5079"} Oct 30 18:45:35.800338 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:35.800161 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:35.842862 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:35.842815 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" podStartSLOduration=1.8428030899999999 podStartE2EDuration="1.84280309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:35.842241148 +0000 UTC m=+806.380433377" watchObservedRunningTime="2026-10-30 18:45:35.84280309 +0000 UTC m=+806.380995302" Oct 30 18:45:38.608078 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:38.608047 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:45:44.396096 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:45:44.396059 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:45:46.808608 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:46.808576 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:46.888026 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:46.887989 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs"] Oct 30 18:45:47.010485 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:47.010452 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-z44ct"] Oct 30 18:45:51.908082 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:51.908042 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" podUID="d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" containerName="payload-processing" containerID="cri-o://1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39" gracePeriod=25 Oct 30 18:45:52.013621 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.013016 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" podUID="c8829315-056f-40bc-8c72-aca7c9e62569" containerName="payload-processing" containerID="cri-o://fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954" gracePeriod=25 Oct 30 18:45:52.563982 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.561108 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:52.749978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.749787 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh\") pod \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " Oct 30 18:45:52.753976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.750682 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials\") pod \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " Oct 30 18:45:52.753976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.752891 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume\") pod \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " Oct 30 18:45:52.753976 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.753037 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume\") pod \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\" (UID: \"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439\") " Oct 30 18:45:52.761990 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.758460 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume" pod "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" (UID: "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:52.762189 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.762139 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume" pod "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" (UID: "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:52.767082 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.763872 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:52.767082 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.763898 6160 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-routing-overlay-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:52.771004 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.770485 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh" pod "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" (UID: "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439"). InnerVolumeSpecName "kube-api-access-rd4dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:52.791369 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.791294 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials" pod "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" (UID: "d0a7c557-fb81-4d7b-8ce3-0b7c5e663439"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:52.838113 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.837871 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:52.865513 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.865487 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd4dh\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-kube-api-access-rd4dh\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:52.865745 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.865731 6160 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439-provider-credentials\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:52.928026 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.927940 6160 generic.go:356] "Generic (PLEG): container finished" podID="c8829315-056f-40bc-8c72-aca7c9e62569" containerID="fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954" exitCode=0 Oct 30 18:45:52.928550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.928210 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" event={"ID":"c8829315-056f-40bc-8c72-aca7c9e62569","Type":"ContainerDied","Data":"fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954"} Oct 30 18:45:52.928550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.928270 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" event={"ID":"c8829315-056f-40bc-8c72-aca7c9e62569","Type":"ContainerDied","Data":"46e64622409d3ab7892230f523837dcbb9c65fb6f8a64fc9b9483ea20d2b5079"} Oct 30 18:45:52.928550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.928308 6160 scope.go:122] "RemoveContainer" containerID="fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954" Oct 30 18:45:52.928550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.928497 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-z44ct" Oct 30 18:45:52.932681 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.932651 6160 generic.go:356] "Generic (PLEG): container finished" podID="d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" containerID="1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39" exitCode=0 Oct 30 18:45:52.932823 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.932725 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" event={"ID":"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439","Type":"ContainerDied","Data":"1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39"} Oct 30 18:45:52.932823 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.932791 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" event={"ID":"d0a7c557-fb81-4d7b-8ce3-0b7c5e663439","Type":"ContainerDied","Data":"6f55c44d1d9dc916c8006a2ab63600098ea4b9f41226f0a234eab7f7317c4bf0"} Oct 30 18:45:52.932942 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.932897 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs" Oct 30 18:45:52.945003 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.944982 6160 scope.go:122] "RemoveContainer" containerID="fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954" Oct 30 18:45:52.945304 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:45:52.945278 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954\": container with ID starting with fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954 not found: ID does not exist" containerID="fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954" Oct 30 18:45:52.945423 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.945314 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954"} err="failed to get container status \"fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954\": rpc error: code = NotFound desc = could not find container \"fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954\": container with ID starting with fa334cf09c6e4e36d5bfeca487e9255133be98eb6a60b1e39b9a774d50b1c954 not found: ID does not exist" Oct 30 18:45:52.945423 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.945338 6160 scope.go:122] "RemoveContainer" containerID="1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39" Oct 30 18:45:52.955485 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.955464 6160 scope.go:122] "RemoveContainer" containerID="1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39" Oct 30 18:45:52.955817 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:45:52.955795 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39\": container with ID starting with 1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39 not found: ID does not exist" containerID="1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39" Oct 30 18:45:52.955909 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.955826 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39"} err="failed to get container status \"1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39\": rpc error: code = NotFound desc = could not find container \"1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39\": container with ID starting with 1c952770c73535d3015c0a19a0ce5d34a3dcacbc0e3ee695bbca896c5c70ba39 not found: ID does not exist" Oct 30 18:45:52.966786 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.966755 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4\") pod \"c8829315-056f-40bc-8c72-aca7c9e62569\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " Oct 30 18:45:52.966941 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.966800 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume\") pod \"c8829315-056f-40bc-8c72-aca7c9e62569\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " Oct 30 18:45:52.967001 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.966978 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials\") pod \"c8829315-056f-40bc-8c72-aca7c9e62569\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " Oct 30 18:45:52.967049 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.967015 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume\") pod \"c8829315-056f-40bc-8c72-aca7c9e62569\" (UID: \"c8829315-056f-40bc-8c72-aca7c9e62569\") " Oct 30 18:45:52.967299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.967265 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume" pod "c8829315-056f-40bc-8c72-aca7c9e62569" (UID: "c8829315-056f-40bc-8c72-aca7c9e62569"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:52.967421 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.967406 6160 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-routing-overlay-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:52.967482 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.967422 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume" pod "c8829315-056f-40bc-8c72-aca7c9e62569" (UID: "c8829315-056f-40bc-8c72-aca7c9e62569"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:52.968911 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.968890 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4" pod "c8829315-056f-40bc-8c72-aca7c9e62569" (UID: "c8829315-056f-40bc-8c72-aca7c9e62569"). InnerVolumeSpecName "kube-api-access-62lw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:52.969070 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.969054 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials" pod "c8829315-056f-40bc-8c72-aca7c9e62569" (UID: "c8829315-056f-40bc-8c72-aca7c9e62569"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:52.994314 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.994275 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs"] Oct 30 18:45:52.998025 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:52.997995 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5d7cd86ff8-98ffs"] Oct 30 18:45:53.068972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.068932 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62lw4\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-kube-api-access-62lw4\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:53.069148 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.068998 6160 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c8829315-056f-40bc-8c72-aca7c9e62569-provider-credentials\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:53.069148 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.069015 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8829315-056f-40bc-8c72-aca7c9e62569-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:45:53.273815 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.273782 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-z44ct"] Oct 30 18:45:53.277777 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.277746 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-z44ct"] Oct 30 18:45:53.578239 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.578209 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8829315-056f-40bc-8c72-aca7c9e62569" path="/var/lib/kubelet/pods/c8829315-056f-40bc-8c72-aca7c9e62569/volumes" Oct 30 18:45:53.578650 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:53.578633 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" path="/var/lib/kubelet/pods/d0a7c557-fb81-4d7b-8ce3-0b7c5e663439/volumes" Oct 30 18:45:59.729533 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:45:59.729502 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:46:08.437887 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:08.437850 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm"] Oct 30 18:46:14.396448 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:14.396410 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:46:14.866940 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:14.866903 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:46:18.760004 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:18.759965 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft"] Oct 30 18:46:20.654224 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:20.654192 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch"] Oct 30 18:46:20.654686 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:20.654386 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" podUID="8ef936eb-defc-410f-9d50-7392e6ad2381" containerName="maas-api" containerID="cri-o://94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85" gracePeriod=30 Oct 30 18:46:20.967449 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:20.967373 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:46:21.057805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.057776 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc\" (UniqueName: \"kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc\") pod \"8ef936eb-defc-410f-9d50-7392e6ad2381\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " Oct 30 18:46:21.057970 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.057882 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs\") pod \"8ef936eb-defc-410f-9d50-7392e6ad2381\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " Oct 30 18:46:21.058035 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.057945 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca\") pod \"8ef936eb-defc-410f-9d50-7392e6ad2381\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " Oct 30 18:46:21.058086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.058057 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\") pod \"8ef936eb-defc-410f-9d50-7392e6ad2381\" (UID: \"8ef936eb-defc-410f-9d50-7392e6ad2381\") " Oct 30 18:46:21.058327 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.058293 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca" pod "8ef936eb-defc-410f-9d50-7392e6ad2381" (UID: "8ef936eb-defc-410f-9d50-7392e6ad2381"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:21.058489 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.058470 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ef936eb-defc-410f-9d50-7392e6ad2381-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:21.060123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.060098 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc" pod "8ef936eb-defc-410f-9d50-7392e6ad2381" (UID: "8ef936eb-defc-410f-9d50-7392e6ad2381"). InnerVolumeSpecName "kube-api-access-kxrnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:21.060233 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.060120 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs" pod "8ef936eb-defc-410f-9d50-7392e6ad2381" (UID: "8ef936eb-defc-410f-9d50-7392e6ad2381"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:21.060233 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.060146 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls" pod "8ef936eb-defc-410f-9d50-7392e6ad2381" (UID: "8ef936eb-defc-410f-9d50-7392e6ad2381"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:21.173978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.163418 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:21.173978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.163456 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxrnc\" (UniqueName: \"kubernetes.io/projected/8ef936eb-defc-410f-9d50-7392e6ad2381-kube-api-access-kxrnc\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:21.173978 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.163470 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ef936eb-defc-410f-9d50-7392e6ad2381-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:21.206225 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.201250 6160 generic.go:356] "Generic (PLEG): container finished" podID="8ef936eb-defc-410f-9d50-7392e6ad2381" containerID="94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85" exitCode=0 Oct 30 18:46:21.206225 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.201396 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" event={"ID":"8ef936eb-defc-410f-9d50-7392e6ad2381","Type":"ContainerDied","Data":"94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85"} Oct 30 18:46:21.206225 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.201451 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" event={"ID":"8ef936eb-defc-410f-9d50-7392e6ad2381","Type":"ContainerDied","Data":"b1c380c796640ada490c89a0419e6adf583763fd09cfce6d9489e9a3ba3fe178"} Oct 30 18:46:21.206225 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.201480 6160 scope.go:122] "RemoveContainer" containerID="94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85" Oct 30 18:46:21.206225 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.201766 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch" Oct 30 18:46:21.248118 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.248091 6160 scope.go:122] "RemoveContainer" containerID="94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85" Oct 30 18:46:21.248521 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:21.248491 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85\": container with ID starting with 94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85 not found: ID does not exist" containerID="94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85" Oct 30 18:46:21.248604 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.248530 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85"} err="failed to get container status \"94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85\": rpc error: code = NotFound desc = could not find container \"94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85\": container with ID starting with 94d3684bf258b88782a63c497d09e9677993b3a6a572caa3c52a672d9fd2cf85 not found: ID does not exist" Oct 30 18:46:21.331712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.331673 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch"] Oct 30 18:46:21.335566 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.335540 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-a47385-76b79768f8-s2cch"] Oct 30 18:46:21.579358 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:21.579317 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ef936eb-defc-410f-9d50-7392e6ad2381" path="/var/lib/kubelet/pods/8ef936eb-defc-410f-9d50-7392e6ad2381/volumes" Oct 30 18:46:23.462038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.461999 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" podUID="bdeebc55-1539-473e-a93a-b19851f0cf90" containerName="main" containerID="cri-o://17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33" gracePeriod=45 Oct 30 18:46:23.728391 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.728330 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:46:23.897783 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.897747 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs\") pod \"bdeebc55-1539-473e-a93a-b19851f0cf90\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " Oct 30 18:46:23.898014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.897805 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home\") pod \"bdeebc55-1539-473e-a93a-b19851f0cf90\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " Oct 30 18:46:23.898014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.897922 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z\" (UniqueName: \"kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z\") pod \"bdeebc55-1539-473e-a93a-b19851f0cf90\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " Oct 30 18:46:23.898014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.897999 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir\") pod \"bdeebc55-1539-473e-a93a-b19851f0cf90\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " Oct 30 18:46:23.898223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898053 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache\") pod \"bdeebc55-1539-473e-a93a-b19851f0cf90\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " Oct 30 18:46:23.898223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898121 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm\") pod \"bdeebc55-1539-473e-a93a-b19851f0cf90\" (UID: \"bdeebc55-1539-473e-a93a-b19851f0cf90\") " Oct 30 18:46:23.898223 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898139 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home" pod "bdeebc55-1539-473e-a93a-b19851f0cf90" (UID: "bdeebc55-1539-473e-a93a-b19851f0cf90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.898386 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898359 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir" pod "bdeebc55-1539-473e-a93a-b19851f0cf90" (UID: "bdeebc55-1539-473e-a93a-b19851f0cf90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.898450 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898387 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache" pod "bdeebc55-1539-473e-a93a-b19851f0cf90" (UID: "bdeebc55-1539-473e-a93a-b19851f0cf90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.898599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898565 6160 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-model-cache\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.898599 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898591 6160 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-home\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.898724 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.898603 6160 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-tmp-dir\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:23.900503 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.900480 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs" pod "bdeebc55-1539-473e-a93a-b19851f0cf90" (UID: "bdeebc55-1539-473e-a93a-b19851f0cf90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:23.900727 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.900705 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm" pod "bdeebc55-1539-473e-a93a-b19851f0cf90" (UID: "bdeebc55-1539-473e-a93a-b19851f0cf90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:23.900799 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.900712 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z" pod "bdeebc55-1539-473e-a93a-b19851f0cf90" (UID: "bdeebc55-1539-473e-a93a-b19851f0cf90"). InnerVolumeSpecName "kube-api-access-4426z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:24.000025 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.999922 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4426z\" (UniqueName: \"kubernetes.io/projected/bdeebc55-1539-473e-a93a-b19851f0cf90-kube-api-access-4426z\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.000025 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.999978 6160 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeebc55-1539-473e-a93a-b19851f0cf90-dshm\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.000025 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:23.999989 6160 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeebc55-1539-473e-a93a-b19851f0cf90-tls-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:24.225947 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.225914 6160 generic.go:356] "Generic (PLEG): container finished" podID="bdeebc55-1539-473e-a93a-b19851f0cf90" containerID="17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33" exitCode=0 Oct 30 18:46:24.226168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.225981 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" event={"ID":"bdeebc55-1539-473e-a93a-b19851f0cf90","Type":"ContainerDied","Data":"17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33"} Oct 30 18:46:24.226168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.226004 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" Oct 30 18:46:24.226168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.226024 6160 scope.go:122] "RemoveContainer" containerID="17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33" Oct 30 18:46:24.226168 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.226014 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm" event={"ID":"bdeebc55-1539-473e-a93a-b19851f0cf90","Type":"ContainerDied","Data":"6055d2af39f41eede45e6399e506a6eb42140cb18f9abbf537ffa32ffe5554ac"} Oct 30 18:46:24.237685 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.237661 6160 scope.go:122] "RemoveContainer" containerID="17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33" Oct 30 18:46:24.238052 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:24.238030 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33\": container with ID starting with 17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33 not found: ID does not exist" containerID="17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33" Oct 30 18:46:24.238161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.238059 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33"} err="failed to get container status \"17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33\": rpc error: code = NotFound desc = could not find container \"17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33\": container with ID starting with 17fe490e8d7d9fc0e04b5bb8091451655146d5add3e2b5423082d680f5eddb33 not found: ID does not exist" Oct 30 18:46:24.277663 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.277593 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm"] Oct 30 18:46:24.289125 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:24.289093 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-b2ef42/premium-w2-b2ef42-kserve-7c5b698c47-gtqkm"] Oct 30 18:46:25.577335 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:25.577303 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdeebc55-1539-473e-a93a-b19851f0cf90" path="/var/lib/kubelet/pods/bdeebc55-1539-473e-a93a-b19851f0cf90/volumes" Oct 30 18:46:32.305065 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:32.304924 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd"] Oct 30 18:46:32.305518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:32.305184 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" containerID="cri-o://9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698" gracePeriod=30 Oct 30 18:46:32.749369 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:32.749283 6160 patch_prober.go:29] interesting pod/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:46:32.749369 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:32.749342 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:46:33.781311 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:33.781248 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" podUID="f2857cfa-12ba-424e-90bd-4f481b977654" containerName="main" containerID="cri-o://8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a" gracePeriod=45 Oct 30 18:46:33.909985 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:33.909932 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" podUID="f2857cfa-12ba-424e-90bd-4f481b977654" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused" Oct 30 18:46:34.037969 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.037896 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:46:34.048804 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.048779 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home\") pod \"f2857cfa-12ba-424e-90bd-4f481b977654\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " Oct 30 18:46:34.049429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.048810 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm\") pod \"f2857cfa-12ba-424e-90bd-4f481b977654\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " Oct 30 18:46:34.049429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.048877 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir\") pod \"f2857cfa-12ba-424e-90bd-4f481b977654\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " Oct 30 18:46:34.049429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.048915 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs\") pod \"f2857cfa-12ba-424e-90bd-4f481b977654\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " Oct 30 18:46:34.049429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.049028 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home" pod "f2857cfa-12ba-424e-90bd-4f481b977654" (UID: "f2857cfa-12ba-424e-90bd-4f481b977654"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.049429 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.049161 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir" pod "f2857cfa-12ba-424e-90bd-4f481b977654" (UID: "f2857cfa-12ba-424e-90bd-4f481b977654"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.049709 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.049691 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache\") pod \"f2857cfa-12ba-424e-90bd-4f481b977654\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " Oct 30 18:46:34.049798 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.049781 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh\" (UniqueName: \"kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh\") pod \"f2857cfa-12ba-424e-90bd-4f481b977654\" (UID: \"f2857cfa-12ba-424e-90bd-4f481b977654\") " Oct 30 18:46:34.050024 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.049997 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache" pod "f2857cfa-12ba-424e-90bd-4f481b977654" (UID: "f2857cfa-12ba-424e-90bd-4f481b977654"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.050273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.050245 6160 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-tmp-dir\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.050273 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.050272 6160 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-model-cache\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.050445 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.050286 6160 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-home\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.051179 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.051151 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm" pod "f2857cfa-12ba-424e-90bd-4f481b977654" (UID: "f2857cfa-12ba-424e-90bd-4f481b977654"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:34.051260 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.051226 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs" pod "f2857cfa-12ba-424e-90bd-4f481b977654" (UID: "f2857cfa-12ba-424e-90bd-4f481b977654"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:34.051796 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.051778 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh" pod "f2857cfa-12ba-424e-90bd-4f481b977654" (UID: "f2857cfa-12ba-424e-90bd-4f481b977654"). InnerVolumeSpecName "kube-api-access-t8sdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:34.151846 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.151816 6160 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2857cfa-12ba-424e-90bd-4f481b977654-dshm\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.151846 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.151844 6160 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2857cfa-12ba-424e-90bd-4f481b977654-tls-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.152067 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.151855 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8sdh\" (UniqueName: \"kubernetes.io/projected/f2857cfa-12ba-424e-90bd-4f481b977654-kube-api-access-t8sdh\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.281014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.280980 6160 generic.go:356] "Generic (PLEG): container finished" podID="f2857cfa-12ba-424e-90bd-4f481b977654" containerID="8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a" exitCode=0 Oct 30 18:46:34.281201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.281018 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" Oct 30 18:46:34.281201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.281067 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" event={"ID":"f2857cfa-12ba-424e-90bd-4f481b977654","Type":"ContainerDied","Data":"8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a"} Oct 30 18:46:34.281201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.281113 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft" event={"ID":"f2857cfa-12ba-424e-90bd-4f481b977654","Type":"ContainerDied","Data":"e12b18fe84a51c3ef076fc09bb7b1d6faad848beca6e1625a3ea0fa351c7010a"} Oct 30 18:46:34.281201 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.281131 6160 scope.go:122] "RemoveContainer" containerID="8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a" Oct 30 18:46:34.290265 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.290250 6160 scope.go:122] "RemoveContainer" containerID="8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a" Oct 30 18:46:34.290520 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:34.290500 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a\": container with ID starting with 8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a not found: ID does not exist" containerID="8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a" Oct 30 18:46:34.290563 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.290529 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a"} err="failed to get container status \"8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a\": rpc error: code = NotFound desc = could not find container \"8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a\": container with ID starting with 8f89e336ad334cbc45b9588341dc0d63948c70dab0206a62d4deda1a451b7c6a not found: ID does not exist" Oct 30 18:46:34.343125 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.343091 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft"] Oct 30 18:46:34.347921 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.347892 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f6f7d5/premium-w0-f6f7d5-kserve-894b4fdf4-xw8ft"] Oct 30 18:46:34.462343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.461611 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v"] Oct 30 18:46:34.462343 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.461835 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" podUID="0d213625-ac20-4e8b-8532-f69d9efd1f83" containerName="maas-api" containerID="cri-o://ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f" gracePeriod=30 Oct 30 18:46:34.764473 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.764446 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:46:34.863017 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.862922 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca\") pod \"0d213625-ac20-4e8b-8532-f69d9efd1f83\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " Oct 30 18:46:34.863448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.863060 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls\") pod \"0d213625-ac20-4e8b-8532-f69d9efd1f83\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " Oct 30 18:46:34.863448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.863121 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs\") pod \"0d213625-ac20-4e8b-8532-f69d9efd1f83\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " Oct 30 18:46:34.863448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.863164 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96\" (UniqueName: \"kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96\") pod \"0d213625-ac20-4e8b-8532-f69d9efd1f83\" (UID: \"0d213625-ac20-4e8b-8532-f69d9efd1f83\") " Oct 30 18:46:34.863448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.863248 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca" pod "0d213625-ac20-4e8b-8532-f69d9efd1f83" (UID: "0d213625-ac20-4e8b-8532-f69d9efd1f83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:34.863634 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.863556 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d213625-ac20-4e8b-8532-f69d9efd1f83-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.865175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.865154 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs" pod "0d213625-ac20-4e8b-8532-f69d9efd1f83" (UID: "0d213625-ac20-4e8b-8532-f69d9efd1f83"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:34.865237 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.865198 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96" pod "0d213625-ac20-4e8b-8532-f69d9efd1f83" (UID: "0d213625-ac20-4e8b-8532-f69d9efd1f83"). InnerVolumeSpecName "kube-api-access-rlj96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:34.865332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.865311 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls" pod "0d213625-ac20-4e8b-8532-f69d9efd1f83" (UID: "0d213625-ac20-4e8b-8532-f69d9efd1f83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:34.965150 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.965057 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.965150 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.965096 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlj96\" (UniqueName: \"kubernetes.io/projected/0d213625-ac20-4e8b-8532-f69d9efd1f83-kube-api-access-rlj96\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:34.965150 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:34.965112 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d213625-ac20-4e8b-8532-f69d9efd1f83-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:35.310647 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.310610 6160 generic.go:356] "Generic (PLEG): container finished" podID="0d213625-ac20-4e8b-8532-f69d9efd1f83" containerID="ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f" exitCode=0 Oct 30 18:46:35.310919 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.310898 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" Oct 30 18:46:35.311062 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.310806 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" event={"ID":"0d213625-ac20-4e8b-8532-f69d9efd1f83","Type":"ContainerDied","Data":"ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f"} Oct 30 18:46:35.311231 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.311212 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v" event={"ID":"0d213625-ac20-4e8b-8532-f69d9efd1f83","Type":"ContainerDied","Data":"c655e62b887f929b0236537546bde2d5c0352ab4300dd92d4e9c8cad018f102b"} Oct 30 18:46:35.311370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.311353 6160 scope.go:122] "RemoveContainer" containerID="ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f" Oct 30 18:46:35.332417 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.332398 6160 scope.go:122] "RemoveContainer" containerID="ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f" Oct 30 18:46:35.332760 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:35.332736 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f\": container with ID starting with ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f not found: ID does not exist" containerID="ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f" Oct 30 18:46:35.332833 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.332774 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f"} err="failed to get container status \"ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f\": rpc error: code = NotFound desc = could not find container \"ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f\": container with ID starting with ed8a83b562480cc7baf8f362876fb42b37ecea5852f0086be2a1d53a3b3aba6f not found: ID does not exist" Oct 30 18:46:35.393178 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.393143 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v"] Oct 30 18:46:35.404505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.404480 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-b2ef42-6d4dc48cb5-nbb8v"] Oct 30 18:46:35.578491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.578416 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d213625-ac20-4e8b-8532-f69d9efd1f83" path="/var/lib/kubelet/pods/0d213625-ac20-4e8b-8532-f69d9efd1f83/volumes" Oct 30 18:46:35.578825 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:35.578810 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2857cfa-12ba-424e-90bd-4f481b977654" path="/var/lib/kubelet/pods/f2857cfa-12ba-424e-90bd-4f481b977654/volumes" Oct 30 18:46:37.544929 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.544903 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:46:37.593667 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593597 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.593667 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593637 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.593866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593715 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.593866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593785 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.593866 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593845 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.594037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593885 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.594037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593907 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.594037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593937 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.594037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593990 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.594037 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.593998 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.594306 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.594067 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs\") pod \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\" (UID: \"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8\") " Oct 30 18:46:37.594554 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.594532 6160 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-credential-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.594636 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.594558 6160 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.594808 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.594765 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.594808 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.594781 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.595151 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.594937 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:37.596814 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.596784 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:37.596922 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.596836 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:37.596922 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.596844 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:37.596922 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.596868 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75" pod "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" (UID: "a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8"). InnerVolumeSpecName "kube-api-access-rnt75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:37.696135 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696107 6160 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-token\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.696135 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696129 6160 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-envoy\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.696135 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696138 6160 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istiod-ca-cert\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.696327 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696145 6160 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-data\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.696327 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696153 6160 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-istio-podinfo\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.696327 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696164 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnt75\" (UniqueName: \"kubernetes.io/projected/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-kube-api-access-rnt75\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:37.696327 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:37.696172 6160 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8-workload-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:38.330316 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.330277 6160 generic.go:356] "Generic (PLEG): container finished" podID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerID="9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698" exitCode=0 Oct 30 18:46:38.330524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.330314 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" event={"ID":"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8","Type":"ContainerDied","Data":"9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698"} Oct 30 18:46:38.330524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.330330 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" Oct 30 18:46:38.330524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.330386 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd" event={"ID":"a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8","Type":"ContainerDied","Data":"c8cd390f0e7d5ee106ea92f42e8332ca4f6e34730db05fe642791fc5feb84fbb"} Oct 30 18:46:38.330524 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.330418 6160 scope.go:122] "RemoveContainer" containerID="9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698" Oct 30 18:46:38.341597 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.341578 6160 scope.go:122] "RemoveContainer" containerID="a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093" Oct 30 18:46:38.354631 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.354607 6160 scope.go:122] "RemoveContainer" containerID="9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698" Oct 30 18:46:38.354962 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:38.354926 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698\": container with ID starting with 9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698 not found: ID does not exist" containerID="9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698" Oct 30 18:46:38.355059 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.354974 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698"} err="failed to get container status \"9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698\": rpc error: code = NotFound desc = could not find container \"9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698\": container with ID starting with 9bde9ad37830461c43ee7337fca27e7fd4e23f53826081ef1e5f71930e806698 not found: ID does not exist" Oct 30 18:46:38.355059 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.354999 6160 scope.go:122] "RemoveContainer" containerID="a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093" Oct 30 18:46:38.355302 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:38.355284 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093\": container with ID starting with a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093 not found: ID does not exist" containerID="a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093" Oct 30 18:46:38.355364 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.355311 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093"} err="failed to get container status \"a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093\": rpc error: code = NotFound desc = could not find container \"a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093\": container with ID starting with a1b711405525a577ab319f1c1dbe0e916b72be883ab95e85c34a3fd8e3af9093 not found: ID does not exist" Oct 30 18:46:38.421275 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.421251 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd"] Oct 30 18:46:38.438966 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:38.438927 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-a47385-openshift-default-69556b768f-hzdxd"] Oct 30 18:46:39.123991 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:39.123151 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:46:39.578462 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:39.578427 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" path="/var/lib/kubelet/pods/a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8/volumes" Oct 30 18:46:40.300716 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:40.300678 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg"] Oct 30 18:46:44.396904 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:44.396867 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:46:44.470936 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:44.468572 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn"] Oct 30 18:46:44.470936 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:44.468868 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" containerID="cri-o://5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252" gracePeriod=30 Oct 30 18:46:44.976649 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:44.976608 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2"] Oct 30 18:46:45.301880 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:45.301820 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerName="payload-pre-processing" containerID="cri-o://64ada98826731da85a91c83f3e044b511a276f708f7a787f624be56a4e7b72d1" gracePeriod=25 Oct 30 18:46:47.342472 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:47.342396 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:49.741075 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.741011 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:46:49.785388 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785353 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.785576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785452 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.785576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785482 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.785576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785513 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.785576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785540 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.785576 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785565 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.786096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785598 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.786096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785630 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.786096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785681 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo\") pod \"fb1cb278-d548-45cd-8919-cc38e0be4554\" (UID: \"fb1cb278-d548-45cd-8919-cc38e0be4554\") " Oct 30 18:46:49.786096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785925 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.786096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.785977 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:49.786096 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.786084 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.786387 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.786152 6160 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb1cb278-d548-45cd-8919-cc38e0be4554-istiod-ca-cert\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.786387 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.786171 6160 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.786387 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.786237 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.786489 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.786450 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.787598 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.787577 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "kube-api-access-tnpsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.788004 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.787980 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.788448 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.788423 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:49.788548 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.788420 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy" pod "fb1cb278-d548-45cd-8919-cc38e0be4554" (UID: "fb1cb278-d548-45cd-8919-cc38e0be4554"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888809 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnpsd\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-kube-api-access-tnpsd\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888848 6160 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-credential-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888870 6160 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-token\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888884 6160 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-data\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888898 6160 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-envoy\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888911 6160 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb1cb278-d548-45cd-8919-cc38e0be4554-workload-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.891973 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:49.888928 6160 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb1cb278-d548-45cd-8919-cc38e0be4554-istio-podinfo\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:50.399710 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.399676 6160 generic.go:356] "Generic (PLEG): container finished" podID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerID="5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252" exitCode=0 Oct 30 18:46:50.399864 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.399737 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" event={"ID":"fb1cb278-d548-45cd-8919-cc38e0be4554","Type":"ContainerDied","Data":"5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252"} Oct 30 18:46:50.399864 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.399743 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" Oct 30 18:46:50.399864 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.399803 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn" event={"ID":"fb1cb278-d548-45cd-8919-cc38e0be4554","Type":"ContainerDied","Data":"8e0c3262645d7cb70699263a6f536ed8511e96ab11c5ac5e1a8a70ee6515eaaa"} Oct 30 18:46:50.399864 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.399824 6160 scope.go:122] "RemoveContainer" containerID="5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252" Oct 30 18:46:50.409773 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.409753 6160 scope.go:122] "RemoveContainer" containerID="fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd" Oct 30 18:46:50.418752 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.418733 6160 scope.go:122] "RemoveContainer" containerID="5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252" Oct 30 18:46:50.419033 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:50.419008 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252\": container with ID starting with 5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252 not found: ID does not exist" containerID="5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252" Oct 30 18:46:50.419125 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.419040 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252"} err="failed to get container status \"5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252\": rpc error: code = NotFound desc = could not find container \"5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252\": container with ID starting with 5bb8ed47912f7e61d107f3a490f3cf2f37964284877a07ddbf6c5ce69eb83252 not found: ID does not exist" Oct 30 18:46:50.419125 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.419058 6160 scope.go:122] "RemoveContainer" containerID="fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd" Oct 30 18:46:50.419344 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:50.419324 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd\": container with ID starting with fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd not found: ID does not exist" containerID="fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd" Oct 30 18:46:50.419390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.419350 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd"} err="failed to get container status \"fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd\": rpc error: code = NotFound desc = could not find container \"fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd\": container with ID starting with fe4d650e5652b19442ca54f17f38b30b19e69b1d2a1f9d2b5f6e5d04f051c2bd not found: ID does not exist" Oct 30 18:46:50.463538 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.463509 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn"] Oct 30 18:46:50.475205 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:50.475178 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-b2ef42-openshift-default-75bc5554f5-4pmzn"] Oct 30 18:46:51.012901 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:51.012869 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf"] Oct 30 18:46:51.013478 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:51.013143 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" containerID="cri-o://98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f" gracePeriod=30 Oct 30 18:46:51.580420 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:51.580383 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" path="/var/lib/kubelet/pods/fb1cb278-d548-45cd-8919-cc38e0be4554/volumes" Oct 30 18:46:51.831316 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:51.831230 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:46:56.268582 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.268557 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:46:56.356927 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.356897 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357098 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.356937 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357098 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357024 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357098 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357084 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357224 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357119 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357224 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357168 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357238 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357289 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357319 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket\") pod \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\" (UID: \"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96\") " Oct 30 18:46:56.357491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357416 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.357491 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357477 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.357737 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357679 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.357825 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357788 6160 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-credential-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.357825 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357804 6160 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.357825 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357811 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:56.357944 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.357818 6160 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-data\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.358411 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.358382 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.359776 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.359748 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "kube-api-access-bn54h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.360038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.359865 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.360226 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.360199 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.360490 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.360460 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo" pod "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" (UID: "a7aee38f-d8ec-4af1-b514-8ee63c0b1b96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:56.443652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.443622 6160 generic.go:356] "Generic (PLEG): container finished" podID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerID="98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f" exitCode=0 Oct 30 18:46:56.443805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.443681 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" Oct 30 18:46:56.443805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.443712 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" event={"ID":"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96","Type":"ContainerDied","Data":"98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f"} Oct 30 18:46:56.443805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.443771 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf" event={"ID":"a7aee38f-d8ec-4af1-b514-8ee63c0b1b96","Type":"ContainerDied","Data":"fc1fa702d184bbf43a16771a2ba274d8da1f0689293a481c53c5e010c27805b2"} Oct 30 18:46:56.443805 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.443794 6160 scope.go:122] "RemoveContainer" containerID="98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f" Oct 30 18:46:56.453014 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.452937 6160 scope.go:122] "RemoveContainer" containerID="a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d" Oct 30 18:46:56.458630 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.458603 6160 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-podinfo\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.458718 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.458634 6160 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istiod-ca-cert\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.458718 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.458649 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn54h\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-kube-api-access-bn54h\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.458718 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.458663 6160 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-token\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.458718 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.458676 6160 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-istio-envoy\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.458718 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.458687 6160 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96-workload-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.461328 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.461302 6160 scope.go:122] "RemoveContainer" containerID="98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f" Oct 30 18:46:56.461571 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:56.461553 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f\": container with ID starting with 98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f not found: ID does not exist" containerID="98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f" Oct 30 18:46:56.461618 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.461577 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f"} err="failed to get container status \"98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f\": rpc error: code = NotFound desc = could not find container \"98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f\": container with ID starting with 98fe9d158c6f849721f39f210beda42afbc8293bab320cfa6ee68287861d5b5f not found: ID does not exist" Oct 30 18:46:56.461618 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.461594 6160 scope.go:122] "RemoveContainer" containerID="a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d" Oct 30 18:46:56.461840 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:56.461822 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d\": container with ID starting with a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d not found: ID does not exist" containerID="a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d" Oct 30 18:46:56.461891 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.461849 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d"} err="failed to get container status \"a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d\": rpc error: code = NotFound desc = could not find container \"a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d\": container with ID starting with a4685c501e724e41f09549a7970e5e3c83e96886b28e46e31897222675a8c27d not found: ID does not exist" Oct 30 18:46:56.496794 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.496773 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf"] Oct 30 18:46:56.503045 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:56.503024 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-brrbf"] Oct 30 18:46:57.342039 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.342007 6160 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:46:57.378515 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.378484 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz"] Oct 30 18:46:57.378714 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.378690 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" podUID="6a4dc8b2-31c9-4776-bef0-61634e28f404" containerName="maas-api" containerID="cri-o://df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880" gracePeriod=30 Oct 30 18:46:57.579238 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.579203 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" path="/var/lib/kubelet/pods/a7aee38f-d8ec-4af1-b514-8ee63c0b1b96/volumes" Oct 30 18:46:57.665946 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.665922 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:46:57.769901 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.769868 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z\" (UniqueName: \"kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z\") pod \"6a4dc8b2-31c9-4776-bef0-61634e28f404\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " Oct 30 18:46:57.770115 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.769923 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs\") pod \"6a4dc8b2-31c9-4776-bef0-61634e28f404\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " Oct 30 18:46:57.770115 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.770103 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca\") pod \"6a4dc8b2-31c9-4776-bef0-61634e28f404\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " Oct 30 18:46:57.770253 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.770144 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\") pod \"6a4dc8b2-31c9-4776-bef0-61634e28f404\" (UID: \"6a4dc8b2-31c9-4776-bef0-61634e28f404\") " Oct 30 18:46:57.770492 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.770458 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca" pod "6a4dc8b2-31c9-4776-bef0-61634e28f404" (UID: "6a4dc8b2-31c9-4776-bef0-61634e28f404"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:57.772121 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.772088 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs" pod "6a4dc8b2-31c9-4776-bef0-61634e28f404" (UID: "6a4dc8b2-31c9-4776-bef0-61634e28f404"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:57.772239 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.772148 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls" pod "6a4dc8b2-31c9-4776-bef0-61634e28f404" (UID: "6a4dc8b2-31c9-4776-bef0-61634e28f404"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:57.772239 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.772184 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z" pod "6a4dc8b2-31c9-4776-bef0-61634e28f404" (UID: "6a4dc8b2-31c9-4776-bef0-61634e28f404"). InnerVolumeSpecName "kube-api-access-k657z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:57.871489 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.871452 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k657z\" (UniqueName: \"kubernetes.io/projected/6a4dc8b2-31c9-4776-bef0-61634e28f404-kube-api-access-k657z\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:57.871489 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.871481 6160 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-metrics-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:57.871489 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.871491 6160 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4dc8b2-31c9-4776-bef0-61634e28f404-openshift-service-ca\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:57.871489 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:57.871499 6160 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a4dc8b2-31c9-4776-bef0-61634e28f404-maas-api-tls\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:46:58.456370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.456330 6160 generic.go:356] "Generic (PLEG): container finished" podID="6a4dc8b2-31c9-4776-bef0-61634e28f404" containerID="df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880" exitCode=0 Oct 30 18:46:58.456793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.456374 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" Oct 30 18:46:58.456793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.456413 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" event={"ID":"6a4dc8b2-31c9-4776-bef0-61634e28f404","Type":"ContainerDied","Data":"df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880"} Oct 30 18:46:58.456793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.456468 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz" event={"ID":"6a4dc8b2-31c9-4776-bef0-61634e28f404","Type":"ContainerDied","Data":"c99c3a2d26f0c61096f294d76d6a9deedfdd1bb6c79e9698149da158dff4c1de"} Oct 30 18:46:58.456793 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.456490 6160 scope.go:122] "RemoveContainer" containerID="df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880" Oct 30 18:46:58.465761 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.465743 6160 scope.go:122] "RemoveContainer" containerID="df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880" Oct 30 18:46:58.466031 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:46:58.466010 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880\": container with ID starting with df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880 not found: ID does not exist" containerID="df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880" Oct 30 18:46:58.466084 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.466039 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880"} err="failed to get container status \"df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880\": rpc error: code = NotFound desc = could not find container \"df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880\": container with ID starting with df5214a04abb9dc5b0593f2f0348fcfda012361de0bf88fc8056a91691115880 not found: ID does not exist" Oct 30 18:46:58.505750 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.505724 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz"] Oct 30 18:46:58.517350 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:58.517324 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a759a1-65d6f96b7f-f2bwz"] Oct 30 18:46:59.577065 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:59.577032 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a4dc8b2-31c9-4776-bef0-61634e28f404" path="/var/lib/kubelet/pods/6a4dc8b2-31c9-4776-bef0-61634e28f404/volumes" Oct 30 18:46:59.997146 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:46:59.997058 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" podUID="6e702456-b696-41e1-b638-b9bb1a344f23" containerName="main" containerID="cri-o://21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac" gracePeriod=45 Oct 30 18:47:00.246865 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.246838 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:47:00.298785 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.298751 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache\") pod \"6e702456-b696-41e1-b638-b9bb1a344f23\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " Oct 30 18:47:00.299100 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.299078 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir\") pod \"6e702456-b696-41e1-b638-b9bb1a344f23\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " Oct 30 18:47:00.299234 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.299200 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache" pod "6e702456-b696-41e1-b638-b9bb1a344f23" (UID: "6e702456-b696-41e1-b638-b9bb1a344f23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.299355 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.299239 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs\") pod \"6e702456-b696-41e1-b638-b9bb1a344f23\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " Oct 30 18:47:00.299476 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.299457 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm\") pod \"6e702456-b696-41e1-b638-b9bb1a344f23\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " Oct 30 18:47:00.299531 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.299518 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home\") pod \"6e702456-b696-41e1-b638-b9bb1a344f23\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " Oct 30 18:47:00.299579 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.299515 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir" pod "6e702456-b696-41e1-b638-b9bb1a344f23" (UID: "6e702456-b696-41e1-b638-b9bb1a344f23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.300048 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.300022 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home" pod "6e702456-b696-41e1-b638-b9bb1a344f23" (UID: "6e702456-b696-41e1-b638-b9bb1a344f23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.300048 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.300048 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx\" (UniqueName: \"kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx\") pod \"6e702456-b696-41e1-b638-b9bb1a344f23\" (UID: \"6e702456-b696-41e1-b638-b9bb1a344f23\") " Oct 30 18:47:00.301456 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.300697 6160 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-model-cache\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.301456 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.300739 6160 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-tmp-dir\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.301456 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.300771 6160 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-home\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.302988 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.302940 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs" pod "6e702456-b696-41e1-b638-b9bb1a344f23" (UID: "6e702456-b696-41e1-b638-b9bb1a344f23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:47:00.304057 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.304035 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm" pod "6e702456-b696-41e1-b638-b9bb1a344f23" (UID: "6e702456-b696-41e1-b638-b9bb1a344f23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.305678 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.305650 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx" pod "6e702456-b696-41e1-b638-b9bb1a344f23" (UID: "6e702456-b696-41e1-b638-b9bb1a344f23"). InnerVolumeSpecName "kube-api-access-6z6hx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:00.402038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.402009 6160 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e702456-b696-41e1-b638-b9bb1a344f23-tls-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.402038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.402031 6160 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e702456-b696-41e1-b638-b9bb1a344f23-dshm\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.402038 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.402040 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z6hx\" (UniqueName: \"kubernetes.io/projected/6e702456-b696-41e1-b638-b9bb1a344f23-kube-api-access-6z6hx\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.468001 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.467966 6160 generic.go:356] "Generic (PLEG): container finished" podID="6e702456-b696-41e1-b638-b9bb1a344f23" containerID="21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac" exitCode=0 Oct 30 18:47:00.468161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.468029 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" Oct 30 18:47:00.468161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.468038 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" event={"ID":"6e702456-b696-41e1-b638-b9bb1a344f23","Type":"ContainerDied","Data":"21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac"} Oct 30 18:47:00.468161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.468082 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2" event={"ID":"6e702456-b696-41e1-b638-b9bb1a344f23","Type":"ContainerDied","Data":"276e5217e500bfd52c7bb7226df831d2a061f3f6b28581c77961e84c4c94e396"} Oct 30 18:47:00.468161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.468093 6160 scope.go:122] "RemoveContainer" containerID="21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac" Oct 30 18:47:00.477454 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.477438 6160 scope.go:122] "RemoveContainer" containerID="21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac" Oct 30 18:47:00.477725 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:47:00.477705 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac\": container with ID starting with 21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac not found: ID does not exist" containerID="21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac" Oct 30 18:47:00.477792 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.477731 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac"} err="failed to get container status \"21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac\": rpc error: code = NotFound desc = could not find container \"21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac\": container with ID starting with 21cb47b0e52b40981c84e6dec9ffa3880ad62ca56f0a474636fa6a9babc1f1ac not found: ID does not exist" Oct 30 18:47:00.517737 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.517712 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2"] Oct 30 18:47:00.522365 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:00.522343 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/premium-w3-0ddc26-kserve-6d5b84df5b-tv2c2"] Oct 30 18:47:01.577464 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:01.577433 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e702456-b696-41e1-b638-b9bb1a344f23" path="/var/lib/kubelet/pods/6e702456-b696-41e1-b638-b9bb1a344f23/volumes" Oct 30 18:47:05.496692 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.496657 6160 generic.go:356] "Generic (PLEG): container finished" podID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerID="64ada98826731da85a91c83f3e044b511a276f708f7a787f624be56a4e7b72d1" exitCode=0 Oct 30 18:47:05.497056 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.496823 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" event={"ID":"77ae66cf-3dc0-451a-a5be-1f2b2e057465","Type":"ContainerDied","Data":"64ada98826731da85a91c83f3e044b511a276f708f7a787f624be56a4e7b72d1"} Oct 30 18:47:05.576896 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.576863 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:47:05.649067 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.649018 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz\" (UniqueName: \"kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz\") pod \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " Oct 30 18:47:05.660390 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.658118 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume\") pod \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\" (UID: \"77ae66cf-3dc0-451a-a5be-1f2b2e057465\") " Oct 30 18:47:05.678804 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.678736 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume" pod "77ae66cf-3dc0-451a-a5be-1f2b2e057465" (UID: "77ae66cf-3dc0-451a-a5be-1f2b2e057465"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:05.683058 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.683024 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz" pod "77ae66cf-3dc0-451a-a5be-1f2b2e057465" (UID: "77ae66cf-3dc0-451a-a5be-1f2b2e057465"). InnerVolumeSpecName "kube-api-access-254nz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:05.765666 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.765628 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-254nz\" (UniqueName: \"kubernetes.io/projected/77ae66cf-3dc0-451a-a5be-1f2b2e057465-kube-api-access-254nz\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:05.765666 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:05.765661 6160 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ae66cf-3dc0-451a-a5be-1f2b2e057465-plugins-config-volume\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:06.504114 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:06.504075 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" event={"ID":"77ae66cf-3dc0-451a-a5be-1f2b2e057465","Type":"ContainerDied","Data":"f28529540a1a7dc9af41f36dd6e9816f396e5d0f1af2595f91aeb330eb0d03e0"} Oct 30 18:47:06.504555 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:06.504135 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg" Oct 30 18:47:06.504555 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:06.504178 6160 scope.go:122] "RemoveContainer" containerID="64ada98826731da85a91c83f3e044b511a276f708f7a787f624be56a4e7b72d1" Oct 30 18:47:06.563210 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:06.563176 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg"] Oct 30 18:47:06.574558 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:06.574530 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a759a1-77d88f6cf9-mqlhg"] Oct 30 18:47:07.579888 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:07.579848 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" path="/var/lib/kubelet/pods/77ae66cf-3dc0-451a-a5be-1f2b2e057465/volumes" Oct 30 18:47:08.508586 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:08.508547 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-dtzmt"] Oct 30 18:47:13.474994 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:13.474478 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt"] Oct 30 18:47:13.476877 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:13.476821 6160 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" containerID="cri-o://80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28" gracePeriod=30 Oct 30 18:47:14.395720 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:47:14.395682 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:47:15.343054 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:15.343021 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rp492_fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e/kuadrant-console-plugin/0.log" Oct 30 18:47:17.337605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:17.337574 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-dtzmt_89b9a796-7be3-4993-a58a-545b4ffc1c0a/limitador/0.log" Oct 30 18:47:18.376153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.376124 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:47:18.726975 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.726917 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:47:18.895850 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.895777 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.895850 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.895834 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.895862 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.895909 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.895975 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896011 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896065 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896342 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896085 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.896342 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896101 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.896342 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896124 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896342 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896190 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.896342 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896200 6160 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token\") pod \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\" (UID: \"c5b0a55b-e180-49d3-a519-c79eabd7d6de\") " Oct 30 18:47:18.896595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896360 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.896595 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896378 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:18.896700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896590 6160 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-data\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.896700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896612 6160 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-certs\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.896700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896623 6160 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istiod-ca-cert\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.896700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896631 6160 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-credential-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.896700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.896639 6160 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-workload-socket\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.898272 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.898245 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:47:18.898272 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.898252 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:18.898570 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.898541 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.898570 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.898542 6160 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr" pod "c5b0a55b-e180-49d3-a519-c79eabd7d6de" (UID: "c5b0a55b-e180-49d3-a519-c79eabd7d6de"). InnerVolumeSpecName "kube-api-access-v6jcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:18.997206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.997174 6160 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-envoy\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.997206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.997202 6160 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-podinfo\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.997206 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.997211 6160 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6jcr\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-kube-api-access-v6jcr\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.997419 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:18.997222 6160 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5b0a55b-e180-49d3-a519-c79eabd7d6de-istio-token\") on node \"ip-10-0-136-103.ec2.internal\" DevicePath \"\"" Oct 30 18:47:19.416636 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.416598 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:47:19.567527 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.567489 6160 generic.go:356] "Generic (PLEG): container finished" podID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerID="80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28" exitCode=0 Oct 30 18:47:19.567719 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.567569 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" event={"ID":"c5b0a55b-e180-49d3-a519-c79eabd7d6de","Type":"ContainerDied","Data":"80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28"} Oct 30 18:47:19.567719 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.567636 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" event={"ID":"c5b0a55b-e180-49d3-a519-c79eabd7d6de","Type":"ContainerDied","Data":"26dc2232a8527053820b423e11584353a34709989fc2d27233a7671e4542dea4"} Oct 30 18:47:19.567719 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.567657 6160 scope.go:122] "RemoveContainer" containerID="80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28" Oct 30 18:47:19.567719 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.567674 6160 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt" Oct 30 18:47:19.577858 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.577833 6160 scope.go:122] "RemoveContainer" containerID="715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee" Oct 30 18:47:19.585833 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.585816 6160 scope.go:122] "RemoveContainer" containerID="80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28" Oct 30 18:47:19.586096 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:47:19.586077 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28\": container with ID starting with 80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28 not found: ID does not exist" containerID="80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28" Oct 30 18:47:19.586153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.586104 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28"} err="failed to get container status \"80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28\": rpc error: code = NotFound desc = could not find container \"80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28\": container with ID starting with 80183f62ecc12c43fc3854a4df0854b3e56968a3e928299ba35e87ff7be18f28 not found: ID does not exist" Oct 30 18:47:19.586153 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.586122 6160 scope.go:122] "RemoveContainer" containerID="715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee" Oct 30 18:47:19.586329 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:47:19.586313 6160 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee\": container with ID starting with 715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee not found: ID does not exist" containerID="715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee" Oct 30 18:47:19.586363 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.586334 6160 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee"} err="failed to get container status \"715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee\": rpc error: code = NotFound desc = could not find container \"715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee\": container with ID starting with 715ed62851eea4a325a2692b7c0c60e156c234de3509c04a499d999f0ab76eee not found: ID does not exist" Oct 30 18:47:19.644442 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.644412 6160 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt"] Oct 30 18:47:19.655814 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:19.655788 6160 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-jdwqt"] Oct 30 18:47:20.148496 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:20.148465 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:47:20.575786 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:20.575760 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:47:21.577420 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:21.577382 6160 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" path="/var/lib/kubelet/pods/c5b0a55b-e180-49d3-a519-c79eabd7d6de/volumes" Oct 30 18:47:32.507779 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:32.507748 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:47:33.475409 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:33.475380 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:47:34.188150 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:34.188115 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:47:34.599626 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:34.599596 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:47:35.173204 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:35.173169 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6f8d6bc878-r4jhp_c4b590be-8d71-4555-a7ae-d86b8e354fcd/maas-api/0.log" Oct 30 18:47:41.682851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:41.682819 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rp492_fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e/kuadrant-console-plugin/0.log" Oct 30 18:47:42.938198 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:42.938168 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-dtzmt_89b9a796-7be3-4993-a58a-545b4ffc1c0a/limitador/0.log" Oct 30 18:47:44.250122 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:44.250092 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:47:44.395569 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:47:44.395528 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:47:44.463051 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:44.463022 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:47:45.492838 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:45.492805 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:47:45.701376 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:45.701343 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:47:46.105518 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:46.105489 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:47:47.135693 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:47.135667 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:47:47.333435 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:47.333405 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:47:48.611090 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:48.611061 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-qtdgm_2775d01f-942d-4bc1-a427-68b71d46324f/main/0.log" Oct 30 18:47:49.209419 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:49.209389 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-xvbrz_2c95c7f7-7920-4be6-96b4-b93b17a36c68/main/0.log" Oct 30 18:47:49.405982 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:49.405938 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8r7qx4_08fa115b-8e06-4399-ab65-657c78368182/main/0.log" Oct 30 18:47:49.587388 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:47:49.587361 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-g98cv_4d1ba668-32cf-47c9-9566-f0de15154a1c/main/0.log" Oct 30 18:48:00.364693 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.364652 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pp8mv/must-gather-fdtjc"] Oct 30 18:48:00.365268 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365246 6160 cpu_manager.go:414] "containerMap: removing container" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365270 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365293 6160 cpu_manager.go:414] "containerMap: removing container" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365301 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365317 6160 cpu_manager.go:414] "containerMap: removing container" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365324 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365337 6160 cpu_manager.go:414] "containerMap: removing container" podUID="d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" containerName="payload-processing" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365345 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" containerName="payload-processing" Oct 30 18:48:00.365357 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365357 6160 cpu_manager.go:414] "containerMap: removing container" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365365 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365376 6160 cpu_manager.go:414] "containerMap: removing container" podUID="f2857cfa-12ba-424e-90bd-4f481b977654" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365383 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2857cfa-12ba-424e-90bd-4f481b977654" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365400 6160 cpu_manager.go:414] "containerMap: removing container" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365407 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365417 6160 cpu_manager.go:414] "containerMap: removing container" podUID="0d213625-ac20-4e8b-8532-f69d9efd1f83" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365423 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d213625-ac20-4e8b-8532-f69d9efd1f83" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365437 6160 cpu_manager.go:414] "containerMap: removing container" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365444 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365457 6160 cpu_manager.go:414] "containerMap: removing container" podUID="8ef936eb-defc-410f-9d50-7392e6ad2381" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365463 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ef936eb-defc-410f-9d50-7392e6ad2381" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365473 6160 cpu_manager.go:414] "containerMap: removing container" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365480 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365491 6160 cpu_manager.go:414] "containerMap: removing container" podUID="c8829315-056f-40bc-8c72-aca7c9e62569" containerName="payload-processing" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365498 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8829315-056f-40bc-8c72-aca7c9e62569" containerName="payload-processing" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365513 6160 cpu_manager.go:414] "containerMap: removing container" podUID="6e702456-b696-41e1-b638-b9bb1a344f23" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365521 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e702456-b696-41e1-b638-b9bb1a344f23" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365535 6160 cpu_manager.go:414] "containerMap: removing container" podUID="bdeebc55-1539-473e-a93a-b19851f0cf90" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365543 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdeebc55-1539-473e-a93a-b19851f0cf90" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365563 6160 cpu_manager.go:414] "containerMap: removing container" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerName="payload-pre-processing" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365570 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerName="payload-pre-processing" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365580 6160 cpu_manager.go:414] "containerMap: removing container" podUID="6a4dc8b2-31c9-4776-bef0-61634e28f404" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365587 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a4dc8b2-31c9-4776-bef0-61634e28f404" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365679 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdeebc55-1539-473e-a93a-b19851f0cf90" containerName="main" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365694 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="77ae66cf-3dc0-451a-a5be-1f2b2e057465" containerName="payload-pre-processing" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365705 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a4dc8b2-31c9-4776-bef0-61634e28f404" containerName="maas-api" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365717 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365727 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365737 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7aee38f-d8ec-4af1-b514-8ee63c0b1b96" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365748 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0a7c557-fb81-4d7b-8ce3-0b7c5e663439" containerName="payload-processing" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365757 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" Oct 30 18:48:00.365751 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365765 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2857cfa-12ba-424e-90bd-4f481b977654" containerName="main" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365773 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365781 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d213625-ac20-4e8b-8532-f69d9efd1f83" containerName="maas-api" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365790 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5b0a55b-e180-49d3-a519-c79eabd7d6de" containerName="istio-proxy" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365799 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365808 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ef936eb-defc-410f-9d50-7392e6ad2381" containerName="maas-api" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365816 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7c2faa6-6f47-4ed2-bd5f-1d0eb97943f8" containerName="istio-proxy" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365825 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8829315-056f-40bc-8c72-aca7c9e62569" containerName="payload-processing" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365834 6160 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e702456-b696-41e1-b638-b9bb1a344f23" containerName="main" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365921 6160 cpu_manager.go:414] "containerMap: removing container" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" Oct 30 18:48:00.367274 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.365930 6160 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb1cb278-d548-45cd-8919-cc38e0be4554" containerName="istio-proxy" Oct 30 18:48:00.369544 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.369525 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.385008 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.384993 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pp8mv\"/\"default-dockercfg-h84sx\"" Oct 30 18:48:00.404299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.404276 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pp8mv\"/\"kube-root-ca.crt\"" Oct 30 18:48:00.404299 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.404290 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pp8mv\"/\"openshift-service-ca.crt\"" Oct 30 18:48:00.406262 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.406235 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp8mv/must-gather-fdtjc"] Oct 30 18:48:00.475009 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.474979 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845-must-gather-output\") pod \"must-gather-fdtjc\" (UID: \"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845\") " pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.475177 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.475147 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbn6\" (UniqueName: \"kubernetes.io/projected/1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845-kube-api-access-vmbn6\") pod \"must-gather-fdtjc\" (UID: \"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845\") " pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.576026 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.576000 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbn6\" (UniqueName: \"kubernetes.io/projected/1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845-kube-api-access-vmbn6\") pod \"must-gather-fdtjc\" (UID: \"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845\") " pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.576161 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.576052 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845-must-gather-output\") pod \"must-gather-fdtjc\" (UID: \"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845\") " pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.576372 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.576358 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845-must-gather-output\") pod \"must-gather-fdtjc\" (UID: \"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845\") " pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.608195 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.608171 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbn6\" (UniqueName: \"kubernetes.io/projected/1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845-kube-api-access-vmbn6\") pod \"must-gather-fdtjc\" (UID: \"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845\") " pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.683986 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.683909 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp8mv/must-gather-fdtjc" Oct 30 18:48:00.977008 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.976929 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp8mv/must-gather-fdtjc"] Oct 30 18:48:00.981337 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:48:00.981310 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1977e7ee_85f0_4ffe_bfd6_0ff3f58cb845.slice/crio-c117c41c3a9cf0f80624620a5b8a12346729ed074d6a8dea3c2ecd129455eb0c WatchSource:0}: Error finding container c117c41c3a9cf0f80624620a5b8a12346729ed074d6a8dea3c2ecd129455eb0c: Status 404 returned error can't find the container with id c117c41c3a9cf0f80624620a5b8a12346729ed074d6a8dea3c2ecd129455eb0c Oct 30 18:48:00.983068 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:00.983049 6160 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:48:01.772215 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:01.772180 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp8mv/must-gather-fdtjc" event={"ID":"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845","Type":"ContainerStarted","Data":"c117c41c3a9cf0f80624620a5b8a12346729ed074d6a8dea3c2ecd129455eb0c"} Oct 30 18:48:04.908535 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:04.908505 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9vhcp_56d694ae-4469-45fe-b66d-36efcb67b5cb/global-pull-secret-syncer/0.log" Oct 30 18:48:06.155683 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:06.155645 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t9b8c_6364988c-a134-41de-9d5f-38728dd0808c/konnectivity-agent/0.log" Oct 30 18:48:06.372910 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:06.372877 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-103.ec2.internal_1e1514c45542b472144184b401445b5d/haproxy/0.log" Oct 30 18:48:06.801966 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:06.801925 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp8mv/must-gather-fdtjc" event={"ID":"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845","Type":"ContainerStarted","Data":"611f8f6e4e8beabbdeb458d399ce69282c900afd9936404c4ec6cadbb3ef12d0"} Oct 30 18:48:06.802149 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:06.801992 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp8mv/must-gather-fdtjc" event={"ID":"1977e7ee-85f0-4ffe-bfd6-0ff3f58cb845","Type":"ContainerStarted","Data":"25e01d7c9e2d4d403b77151a4d9502ac56e0b7d2e758836874014be203540486"} Oct 30 18:48:06.842269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:06.842226 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-pp8mv/must-gather-fdtjc" podStartSLOduration=2.102405258 podStartE2EDuration="6.842213419s" totalImagesPullingTime="4.739808161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:06.840754631 +0000 UTC m=+957.378946835" watchObservedRunningTime="2026-10-30 18:48:06.842213419 +0000 UTC m=+957.380405664" Oct 30 18:48:07.737592 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:07.737560 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:07.839086 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:07.839058 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:08.154117 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:08.154085 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:08.210559 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:08.210523 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:08.345175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:08.345150 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:08.390927 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:08.390894 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:08.674639 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:08.674614 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:08.918348 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:08.918314 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:09.857228 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:09.857201 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:10.133193 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:10.133115 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:10.230565 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:10.230529 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:10.279972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:10.279936 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:10.511901 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:10.511832 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:10.564123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:10.564084 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:10.724196 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:10.724169 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:11.008836 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:11.008808 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:11.981928 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:11.981900 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:12.270193 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:12.270108 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:12.343203 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:12.343166 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:12.394434 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:12.394406 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:12.691040 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:12.690992 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:12.740389 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:12.740334 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:12.831068 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:12.831039 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:13.083725 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:13.083692 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:14.397162 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:48:14.397126 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:48:14.849175 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:14.849134 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:14.883567 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:14.883539 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:15.080094 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.080061 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:15.110170 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.110097 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:15.196132 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.196096 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:15.254710 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.254673 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:15.385218 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.385138 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:15.412732 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.412706 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:15.477766 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.477728 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:15.604904 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.604870 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:15.723833 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.723754 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:15.788829 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.788800 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:15.925107 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.925078 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:15.982278 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:15.982188 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:18.609370 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:18.609339 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rp492_fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e/kuadrant-console-plugin/0.log" Oct 30 18:48:18.771457 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:18.771425 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rp492_fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e/kuadrant-console-plugin/0.log" Oct 30 18:48:18.855087 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:18.855053 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rp492_fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e/kuadrant-console-plugin/0.log" Oct 30 18:48:19.122612 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:19.122582 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-dtzmt_89b9a796-7be3-4993-a58a-545b4ffc1c0a/limitador/0.log" Oct 30 18:48:19.491849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:19.491772 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-dtzmt_89b9a796-7be3-4993-a58a-545b4ffc1c0a/limitador/0.log" Oct 30 18:48:19.647550 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:19.647517 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-dtzmt_89b9a796-7be3-4993-a58a-545b4ffc1c0a/limitador/0.log" Oct 30 18:48:20.113608 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.113576 6160 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s"] Oct 30 18:48:20.126103 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.126076 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.140134 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.140111 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s"] Oct 30 18:48:20.156616 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.156593 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tq9m\"/\"openshift-service-ca.crt\"" Oct 30 18:48:20.156616 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.156602 6160 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tq9m\"/\"kube-root-ca.crt\"" Oct 30 18:48:20.171048 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.171025 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-proc\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.171156 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.171055 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55c58\" (UniqueName: \"kubernetes.io/projected/8461e3b4-0c8f-48b0-832d-01a69cdff03e-kube-api-access-55c58\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.171156 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.171087 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-podres\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.171295 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.171274 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-sys\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.171332 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.171309 6160 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-lib-modules\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.178700 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.178681 6160 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6tq9m\"/\"default-dockercfg-vlv4w\"" Oct 30 18:48:20.272605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272569 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-proc\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272605 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272607 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55c58\" (UniqueName: \"kubernetes.io/projected/8461e3b4-0c8f-48b0-832d-01a69cdff03e-kube-api-access-55c58\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272650 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-podres\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272688 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-proc\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272713 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-sys\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272746 6160 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-lib-modules\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272754 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-podres\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.272849 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272831 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-lib-modules\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.273091 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.272854 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8461e3b4-0c8f-48b0-832d-01a69cdff03e-sys\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.295383 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.295350 6160 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55c58\" (UniqueName: \"kubernetes.io/projected/8461e3b4-0c8f-48b0-832d-01a69cdff03e-kube-api-access-55c58\") pod \"perf-node-gather-daemonset-b4t2s\" (UID: \"8461e3b4-0c8f-48b0-832d-01a69cdff03e\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.438012 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.437900 6160 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.733389 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.733252 6160 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s"] Oct 30 18:48:20.742988 ip-10-0-136-103 kubenswrapper[6160]: W1030 18:48:20.737673 6160 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8461e3b4_0c8f_48b0_832d_01a69cdff03e.slice/crio-5b0c734d93a7e2418909908c87bf1310dc5f13860a48203268a95e1b5b0f8aac WatchSource:0}: Error finding container 5b0c734d93a7e2418909908c87bf1310dc5f13860a48203268a95e1b5b0f8aac: Status 404 returned error can't find the container with id 5b0c734d93a7e2418909908c87bf1310dc5f13860a48203268a95e1b5b0f8aac Oct 30 18:48:20.878513 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.878479 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" event={"ID":"8461e3b4-0c8f-48b0-832d-01a69cdff03e","Type":"ContainerStarted","Data":"7612339ce13a5b41f00fb8c2f4eec39b740c2add1aafa3b892c2043a3935c3f3"} Oct 30 18:48:20.878643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.878536 6160 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" event={"ID":"8461e3b4-0c8f-48b0-832d-01a69cdff03e","Type":"ContainerStarted","Data":"5b0c734d93a7e2418909908c87bf1310dc5f13860a48203268a95e1b5b0f8aac"} Oct 30 18:48:20.878643 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.878561 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:20.918291 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:20.918094 6160 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" podStartSLOduration=0.918077189 podStartE2EDuration="918.077189ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:20.917197349 +0000 UTC m=+971.455389553" watchObservedRunningTime="2026-10-30 18:48:20.918077189 +0000 UTC m=+971.456269408" Oct 30 18:48:21.031712 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.031686 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/alertmanager/0.log" Oct 30 18:48:21.134126 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.134089 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/config-reloader/0.log" Oct 30 18:48:21.238191 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.238167 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/kube-rbac-proxy-web/0.log" Oct 30 18:48:21.343290 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.343202 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/kube-rbac-proxy/0.log" Oct 30 18:48:21.460652 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.460577 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/kube-rbac-proxy-metric/0.log" Oct 30 18:48:21.576018 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.575989 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/prom-label-proxy/0.log" Oct 30 18:48:21.680218 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.680122 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f2c92ce6-75ab-4c1e-bf93-d15d70abd3f9/init-config-reloader/0.log" Oct 30 18:48:21.900092 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.900038 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-85r7x_513b3b1c-2fd2-42df-9438-4f7e52bf43bc/kube-state-metrics/0.log" Oct 30 18:48:21.999109 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:21.999003 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-85r7x_513b3b1c-2fd2-42df-9438-4f7e52bf43bc/kube-rbac-proxy-main/0.log" Oct 30 18:48:22.106696 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:22.106648 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-85r7x_513b3b1c-2fd2-42df-9438-4f7e52bf43bc/kube-rbac-proxy-self/0.log" Oct 30 18:48:22.328324 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:22.328296 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-cft4b_2c8123ce-1ece-4d21-8bcb-4c7effd34d1d/monitoring-plugin/0.log" Oct 30 18:48:23.094684 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:23.094654 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s76gv_bbe9260b-f403-4983-8e9a-6c328064b4c1/node-exporter/0.log" Oct 30 18:48:23.195968 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:23.195937 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s76gv_bbe9260b-f403-4983-8e9a-6c328064b4c1/kube-rbac-proxy/0.log" Oct 30 18:48:23.297547 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:23.297518 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s76gv_bbe9260b-f403-4983-8e9a-6c328064b4c1/init-textfile/0.log" Oct 30 18:48:24.539864 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:24.539828 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-rk24f_1eb83f0f-6234-4239-8065-df0c41d41b32/prometheus-operator/0.log" Oct 30 18:48:24.655477 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:24.655448 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-rk24f_1eb83f0f-6234-4239-8065-df0c41d41b32/kube-rbac-proxy/0.log" Oct 30 18:48:24.952186 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:24.952113 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55f45f79df-6q4pk_c68a11ea-7b0c-414c-a1dd-a39b3b89472e/telemeter-client/0.log" Oct 30 18:48:25.081670 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:25.081636 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55f45f79df-6q4pk_c68a11ea-7b0c-414c-a1dd-a39b3b89472e/reload/0.log" Oct 30 18:48:25.185295 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:25.185268 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55f45f79df-6q4pk_c68a11ea-7b0c-414c-a1dd-a39b3b89472e/kube-rbac-proxy/0.log" Oct 30 18:48:26.892498 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:26.892470 6160 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-b4t2s" Oct 30 18:48:26.894571 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:26.894550 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-h244v_ee9b04a4-387d-4702-b4ad-9a273642e191/manager/0.log" Oct 30 18:48:26.952851 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:26.952826 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-h244v_ee9b04a4-387d-4702-b4ad-9a273642e191/copy-manifests/0.log" Oct 30 18:48:27.110746 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:27.110720 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-h244v_ee9b04a4-387d-4702-b4ad-9a273642e191/manager/0.log" Oct 30 18:48:27.167031 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:27.165886 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-h244v_ee9b04a4-387d-4702-b4ad-9a273642e191/copy-manifests/0.log" Oct 30 18:48:27.964379 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:27.964333 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:28.189815 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.189787 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7b4cb8955-jcvnm_59d43b61-fbe2-4332-beac-45f26bf72788/manager/0.log" Oct 30 18:48:28.294393 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.294366 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:28.346972 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.346920 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:28.498656 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.498625 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/0.log" Oct 30 18:48:28.553336 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.553267 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5b87b589-7fw2t_486bbdb8-94a4-423d-91ef-3e7da516e56b/manager/1.log" Oct 30 18:48:28.673677 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.673651 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:28.842389 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:28.842308 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-rxctj_fc434352-f592-4d62-b816-25a0786a696f/proxy/0.log" Oct 30 18:48:30.210532 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.210493 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:30.250990 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.250960 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:30.329077 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.329034 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:30.367505 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.367471 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:30.553619 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.553573 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:30.606464 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.606436 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:30.642537 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.642513 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:30.701269 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.701238 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:30.728012 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.727981 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:30.820255 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:30.820166 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:31.045858 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:31.045822 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:31.104583 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:31.104513 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:31.133258 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:31.133233 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:31.187433 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:31.187407 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:32.842501 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:32.840930 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sxvp2_a1a0af93-464e-4b58-bf1c-ab9de497715d/dns/0.log" Oct 30 18:48:32.944565 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:32.944541 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sxvp2_a1a0af93-464e-4b58-bf1c-ab9de497715d/kube-rbac-proxy/0.log" Oct 30 18:48:33.160845 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:33.160769 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kbnmf_7eb17ce4-c80f-476b-8682-c1a329bd82fb/dns-node-resolver/0.log" Oct 30 18:48:34.691238 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:34.691200 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z8j6q_422a1ecf-21d1-433f-8ede-d9de5a5940b7/node-ca/0.log" Oct 30 18:48:36.871924 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:36.871887 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:36.952430 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:36.952400 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:37.503858 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:37.503827 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:37.611160 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:37.611127 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:37.620722 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:37.620699 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:37.657123 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:37.657094 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:37.824929 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:37.824901 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:37.948138 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:37.948104 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:38.007049 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:38.007018 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:38.129556 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:38.129471 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:38.385257 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:38.385148 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:38.441089 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:38.441056 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:38.488526 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:38.488496 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:38.498439 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:38.498410 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:40.169120 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:40.169091 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rp492_fe3f5e6e-42ae-4baa-bb2e-56f3bf15836e/kuadrant-console-plugin/0.log" Oct 30 18:48:40.337674 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:40.337636 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mbxqk_048e9269-1abc-446c-baed-e789c6e91df0/serve-healthcheck-canary/0.log" Oct 30 18:48:40.424083 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:40.424015 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-dtzmt_89b9a796-7be3-4993-a58a-545b4ffc1c0a/limitador/0.log" Oct 30 18:48:41.083774 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.083722 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-nw46b_d08f0f8f-8ee7-4566-badb-cf1f043eb004/discovery/0.log" Oct 30 18:48:41.127664 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.127640 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869444d94f-84r4k_2860dd5f-fbe9-4e39-8910-e448f2d17cc0/kube-auth-proxy/0.log" Oct 30 18:48:41.428899 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.428811 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-q9ng9_f3578481-4db4-4032-9949-49c93f833fe8/istio-proxy/0.log" Oct 30 18:48:41.491706 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.491679 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qnfsg_54ce7764-e6ec-45fd-b490-51c7c5f531f5/istio-proxy/0.log" Oct 30 18:48:41.615226 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.615199 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tbzd6_d7fe0720-634c-4285-9b18-9a061f992421/kube-rbac-proxy/0.log" Oct 30 18:48:41.619250 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.619226 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tqkjg_2c4b38b0-ea36-4f64-9e8e-927edf4a45e8/istio-proxy/0.log" Oct 30 18:48:41.714677 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.714608 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tbzd6_d7fe0720-634c-4285-9b18-9a061f992421/exporter/0.log" Oct 30 18:48:41.814713 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.814679 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tbzd6_d7fe0720-634c-4285-9b18-9a061f992421/extractor/0.log" Oct 30 18:48:41.943121 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:41.943092 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-n2ljx_44a75941-102f-406b-8e35-d43c8fb93408/payload-pre-processing/0.log" Oct 30 18:48:42.004711 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:42.004642 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-b8qh5_720c779d-e253-4480-b411-75e6ca6e3dbc/payload-processing/0.log" Oct 30 18:48:44.398974 ip-10-0-136-103 kubenswrapper[6160]: E1030 18:48:44.398928 6160 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tbzd6" containerName="extractor" Oct 30 18:48:45.406511 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:45.406474 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-h244v_ee9b04a4-387d-4702-b4ad-9a273642e191/manager/0.log" Oct 30 18:48:45.509129 ip-10-0-136-103 kubenswrapper[6160]: I1030 18:48:45.509101 6160 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-h244v_ee9b04a4-387d-4702-b4ad-9a273642e191/copy-manifests/0.log"