Nov 04 01:33:47.836922 ip-10-0-140-18 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:33:48.254428 ip-10-0-140-18 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:33:48.254965 ip-10-0-140-18 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:33:48.254965 ip-10-0-140-18 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:33:48.254965 ip-10-0-140-18 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:33:48.259697 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259582    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.259697 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259700    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259704    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259707    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259711    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259714    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259716    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259719    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259722    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259725    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259728    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259730    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259733    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259735    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259738    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259741    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259743    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259746    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259749    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259751    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.259776 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259753    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259756    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259758    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259761    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259770    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259772    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259775    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259777    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259780    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259783    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259786    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259788    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259791    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259795    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259797    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259800    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259802    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259805    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259807    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259809    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.260236 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259812    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259814    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259817    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259819    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259822    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259825    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259827    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259829    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259831    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259834    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259836    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259838    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259841    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259844    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259847    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259850    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259852    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259854    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259857    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.260702 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259861    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259866    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259869    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259871    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259874    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259876    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259879    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259882    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259884    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259887    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259889    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259892    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259894    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259897    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259899    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259901    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259904    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259906    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259909    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259912    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.261159 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259914    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259917    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259919    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259921    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259924    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259926    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259929    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259932    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259935    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259937    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259940    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259943    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259945    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259947    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259950    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259952    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259955    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259957    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259960    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.261637 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259964    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259968    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259971    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259974    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259977    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259979    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259981    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.259984    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260828    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260835    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260838    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260842    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260848    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260851    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260854    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260857    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260859    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260862    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260864    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260867    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.262090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260871    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260874    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260876    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260879    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260882    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260884    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260887    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260889    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260892    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260894    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260897    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260899    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260902    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260904    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260907    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260909    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260912    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260914    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260916    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.262672 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260919    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260921    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260924    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260926    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260929    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260931    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260935    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260938    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260940    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260942    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260945    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260948    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260950    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260952    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260955    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260957    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260959    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260962    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260964    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.263123 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260966    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260969    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260971    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260973    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260976    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260978    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260980    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260983    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260985    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260987    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260990    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260992    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260995    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260997    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.260999    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261002    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261004    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261007    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261009    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.263555 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261015    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261018    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261020    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261025    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261028    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261032    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261034    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261037    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261039    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261042    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261045    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261047    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261049    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261051    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261054    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261056    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261059    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261061    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261064    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.264043 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261066    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261068    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261071    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261073    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261076    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261078    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261080    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261082    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261085    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261087    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261090    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261092    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261095    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261098    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261101    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261109    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261112    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261114    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261665    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261672    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.264482 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261675    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261678    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261681    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261683    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261686    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261689    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261691    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261694    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261696    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261699    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261701    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261703    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261706    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261708    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261711    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261713    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261716    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261718    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261721    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261723    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.264962 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261725    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261728    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261730    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261735    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261737    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261742    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261744    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261752    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261755    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261758    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261760    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261763    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261765    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261767    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261770    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261772    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261774    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261777    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261779    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261782    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.265411 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261784    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261787    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261789    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261791    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261794    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261796    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261798    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261801    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261803    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261805    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261808    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261810    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261813    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261815    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261818    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261821    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261824    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261827    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261830    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261832    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.265905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261840    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261843    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261845    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261847    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261850    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261853    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261855    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261857    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261860    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261862    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261865    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261867    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261869    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261872    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261874    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261877    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261879    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261881    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261884    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.266351 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261888    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261891    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261894    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261896    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261899    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261901    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261904    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261906    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261910    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261913    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261917    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261920    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261923    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261925    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261928    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261931    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261933    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261936    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261939    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.266786 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261942    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261944    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261947    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261949    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261952    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.261954    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262934    2576 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262948    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262956    2576 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262961    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262965    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262968    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262972    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262977    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262980    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262983    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262986    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262989    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262992    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262995    2576 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.262998    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263002    2576 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263005    2576 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:33:48.267221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263008    2576 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263013    2576 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263016    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263019    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263022    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263028    2576 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263032    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263035    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263038    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263042    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263044    2576 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263047    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263050    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263053    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263056    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263061    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263063    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263066    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263069    2576 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263072    2576 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263075    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263081    2576 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263085    2576 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263087    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:33:48.267741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263091    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263093    2576 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263097    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263100    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263103    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263106    2576 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263109    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263112    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263115    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263119    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263122    2576 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263125    2576 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263128    2576 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263133    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263137    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263140    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263143    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263147    2576 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263150    2576 flags.go:64] FLAG: --help="false"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263153    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263156    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263159    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263161    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:33:48.268326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263165    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263168    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263171    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263173    2576 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263176    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263179    2576 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263182    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263185    2576 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263187    2576 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263190    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263193    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263196    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263199    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263201    2576 flags.go:64] FLAG: --lock-file=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263205    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263208    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263211    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263217    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263219    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263222    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263225    2576 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263228    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263231    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263234    2576 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263237    2576 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:33:48.268867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263242    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263245    2576 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263249    2576 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263252    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263255    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263258    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263260    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263263    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263266    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263269    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263276    2576 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263278    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263281    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263284    2576 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263287    2576 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263290    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263292    2576 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263295    2576 flags.go:64] FLAG: --port="10250"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263298    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263301    2576 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0879c243918a54fd4"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263304    2576 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263306    2576 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263309    2576 flags.go:64] FLAG: --register-node="true"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263312    2576 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263316    2576 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263319    2576 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:33:48.269466 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263322    2576 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263324    2576 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263328    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263331    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263334    2576 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263337    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263340    2576 flags.go:64] FLAG: --runonce="false"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263343    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263346    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263349    2576 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263351    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263354    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263357    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263360    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263363    2576 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263366    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263368    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263371    2576 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263374    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263376    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263379    2576 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263382    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263387    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263390    2576 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263393    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:33:48.270230 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263397    2576 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263400    2576 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263403    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263407    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263411    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263414    2576 flags.go:64] FLAG: --v="2"
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263418    2576 flags.go:64] FLAG: --version="false"
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263423    2576 flags.go:64] FLAG: --vmodule=""
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263427    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263430    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263602    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263607    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263610    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263612    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263615    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263618    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263620    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263623    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263625    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263628    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263631    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263633    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.271249 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263636    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263638    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263640    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263643    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263645    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263648    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263650    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263653    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263655    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263657    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263660    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263663    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263665    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263669    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263672    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263675    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263677    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263680    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263682    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263685    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.272150 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263687    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263690    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263693    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263695    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263697    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263700    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263702    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263705    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263707    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263710    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263712    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263715    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263717    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263720    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263722    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263724    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263727    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263729    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263731    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.272986 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263734    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263736    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263738    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263741    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263743    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263746    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263749    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263753    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263756    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263759    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263761    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263764    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263766    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263769    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263771    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263774    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263776    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263779    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263781    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.273903 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263785    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263787    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263789    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263792    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263794    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263797    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263799    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263801    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263804    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263806    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263809    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263811    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263813    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263816    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263818    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263821    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263825    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263828    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263831    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.274724 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263835    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263839    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263841    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263845    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263849    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263851    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263854    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263857    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263860    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263863    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263865    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263868    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263870    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263872    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263875    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263878    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.275163 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263880    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.263907    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263943    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263947    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263951    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263954    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263957    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263960    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263962    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263965    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263967    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263969    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263972    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263974    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263977    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263979    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.275694 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263983    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263986    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263989    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263992    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263994    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263997    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.263999    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264002    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264004    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264007    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264009    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264011    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264014    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264017    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264020    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264024    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264028    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264030    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264033    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.276305 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264035    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264039    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264042    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264045    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264047    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264050    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264053    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264056    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264058    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264061    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264063    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264065    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264068    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264071    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264074    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264077    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264079    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264082    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264084    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264086    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.277000 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264089    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264091    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264093    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264096    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264099    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264101    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264103    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264106    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264108    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264111    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264113    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264116    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264118    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264123    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264125    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264128    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264130    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264133    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264136    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264138    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.277462 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264140    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264142    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264145    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264147    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264149    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264154    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264156    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264160    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264163    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264165    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264168    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264170    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264172    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264175    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264177    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264179    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264182    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264185    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264188    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264190    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.278003 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264192    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264194    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264197    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264200    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264202    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264204    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264207    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264209    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264211    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264214    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264216    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264218    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.264221    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.272473    2576 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.272498    2576 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.272521    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.272529    2576 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272642    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272651    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.278726 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272657    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272662    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272666    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272671    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272675    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272679    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272683    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272687    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272691    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272695    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272699    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272704    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272708    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272712    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272716    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272720    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272724    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272728    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272732    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272737    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.279359 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272741    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272745    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272753    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272760    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272766    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272771    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272776    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272780    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272785    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272789    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272794    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272798    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272802    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272806    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272812    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272816    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272820    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272824    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272828    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.280064 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272832    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272836    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272841    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272845    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272849    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272853    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272858    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272861    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272867    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272871    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272876    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272880    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272884    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272888    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272892    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272896    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272901    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272905    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272909    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.280621 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272913    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272917    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272922    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272926    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272930    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272935    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272939    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272943    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272946    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272951    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272955    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272959    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272963    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272967    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272971    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272975    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272979    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272982    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272986    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272989    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.281120 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272993    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.272997    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273001    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273005    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273009    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273014    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273018    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273022    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273026    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273030    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273035    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273039    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273043    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273048    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273052    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273057    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273061    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273067    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273073    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.281736 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273077    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273081    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273086    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273089    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273093    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273098    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273102    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.273144    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273376    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273383    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273388    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273393    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273397    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273401    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273406    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273410    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.282201 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273414    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273419    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273423    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273427    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273431    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273435    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273439    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273443    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273452    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273458    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273463    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273467    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273472    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273476    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273480    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273484    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273488    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273493    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273497    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.282665 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273501    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273505    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273509    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273513    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273517    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273521    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273525    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273529    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273546    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273550    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273555    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273558    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273579    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273584    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273587    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273590    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273594    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273598    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273602    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.283097 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273606    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273610    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273613    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273618    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273622    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273626    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273630    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273633    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273637    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273641    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273644    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273648    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273651    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273655    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273659    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273663    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273667    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273671    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273675    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273679    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.283779 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273682    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273687    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273690    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273694    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273698    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273702    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273708    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273715    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273719    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273723    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273727    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273732    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273736    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273740    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273743    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273747    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273753    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273757    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273761    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273765    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.284466 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273769    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273773    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273777    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273781    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273784    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273789    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273793    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273797    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273801    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273805    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273810    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273814    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273818    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273822    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273826    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273830    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273834    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273838    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273842    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.285013 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273846    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.273893    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273943    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273949    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273954    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273958    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273962    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273966    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273970    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273974    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273979    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273983    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273987    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273991    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.273995    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:33:48.285744 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274001    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274006    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274010    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274014    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274018    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274022    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274027    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274031    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274034    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274038    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274042    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274049    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274053    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274057    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274061    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274066    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274070    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274074    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274078    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274082    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:33:48.286126 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274086    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274090    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274094    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274098    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274102    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274106    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274110    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274114    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274119    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274123    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274128    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274132    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274135    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274139    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274143    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274147    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274152    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274156    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274160    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274164    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:33:48.286599 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274168    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274172    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274177    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274184    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274188    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274193    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274199    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274204    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274208    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274212    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274218    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274225    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274231    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274236    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274240    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274245    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274249    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274254    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274258    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:33:48.287072 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274262    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274266    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274271    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274275    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274279    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274283    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274287    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274291    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274295    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274299    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274303    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274307    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274311    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274314    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274318    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274322    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274328    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274332    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274336    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:33:48.287527 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274340    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274345    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274348    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274354    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274358    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274362    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274367    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274370    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274374    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274379    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274383    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274387    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274391    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274395    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:48.274399    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.274484    2576 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.278144    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.280927    2576 server.go:1014] "Starting client certificate rotation"
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.281036    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:33:48.287983 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.281644    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:33:48.288411 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.283967    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:33:48.288411 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.285096    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:33:48.297625 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.297606    2576 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:33:48.301592 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.301561    2576 log.go:25] "Validated CRI v1 image API"
Nov 04 01:33:48.302376 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.302353    2576 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:33:48.306463 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.306444    2576 fs.go:135] Filesystem UUIDs: map[1b47b089-e7a6-40dc-a2c2-d502dce1ede9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a790cc8d-25aa-4c50-8fe4-bed5278eaf2c:/dev/nvme0n1p4]
Nov 04 01:33:48.306527 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.306463    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 01:33:48.312786 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.312668    2576 manager.go:217] Machine: {Timestamp:2026-11-04 01:33:48.311077242 +0000 UTC m=+0.351005347 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499996 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25a3dd573d45a17c348650a16ffb53 SystemUUID:ec25a3dd-573d-45a1-7c34-8650a16ffb53 BootID:f5b3140f-d33f-431d-b311-564fa6426591 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:72:5b:4b:f8:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:72:5b:4b:f8:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:7c:3c:22:7b:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:33:48.312786 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.312783    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:33:48.312891 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.312876    2576 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:33:48.315946 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.315915    2576 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:33:48.316124 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.315947    2576 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-18.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:33:48.316178 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.316127    2576 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:33:48.316178 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.316135    2576 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:33:48.316178 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.316165    2576 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:33:48.316255 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.316184    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:33:48.317509 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.317499    2576 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:33:48.321795 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.321782    2576 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:33:48.321918 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.321910    2576 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:33:48.322700 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.322690    2576 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:33:48.322731 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.322705    2576 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:33:48.322731 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.322716    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:33:48.322731 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.322725    2576 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:33:48.322836 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.322733    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:33:48.323161 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.323145    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:33:48.325679 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.325663    2576 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:33:48.333608 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.333586    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:33:48.335484 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335468    2576 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:33:48.335589 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335509    2576 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:33:48.335736 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335723    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:33:48.335798 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335744    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:33:48.335798 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335755    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:33:48.335798 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335765    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:33:48.335798 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335776    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:33:48.335798 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.335784    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:33:48.336986 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.336976    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:33:48.337046 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.336991    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:33:48.337046 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.337004    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:33:48.337046 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.337014    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:33:48.337046 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.337029    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:33:48.337046 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.337042    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:33:48.337690 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.337678    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:33:48.338909 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.337694    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:33:48.342195 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.342174    2576 server.go:1263] "Started kubelet"
Nov 04 01:33:48.342288 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.342246    2576 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:33:48.342340 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.342254    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:33:48.342340 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.342319    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:33:48.343087 ip-10-0-140-18 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:33:48.343858 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.343841    2576 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:33:48.344867 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.344851    2576 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:33:48.348153 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.348135    2576 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 01:33:48.350011 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.349996    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:33:48.351170 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.351157    2576 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:33:48.351266 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.351250    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:48.351880 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.351861    2576 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:33:48.351880 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.351882    2576 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:33:48.352008 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.351903    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:33:48.352070 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352045    2576 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:33:48.352070 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352066    2576 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:33:48.352306 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352295    2576 factory.go:153] Registering CRI-O factory
Nov 04 01:33:48.352343 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352328    2576 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:33:48.352377 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352369    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:33:48.352418 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352378    2576 factory.go:55] Registering systemd factory
Nov 04 01:33:48.352418 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352383    2576 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:33:48.352418 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352396    2576 factory.go:103] Registering Raw factory
Nov 04 01:33:48.352418 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.352405    2576 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:33:48.353296 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.353282    2576 manager.go:319] Starting recovery of all containers
Nov 04 01:33:48.365060 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.365046    2576 manager.go:324] Recovery completed
Nov 04 01:33:48.373050 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.373032    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.374651 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.374634    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.374716 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.374665    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.374716 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.374674    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.375175 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375163    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.375175 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375171    2576 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:33:48.375265 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375178    2576 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:33:48.375265 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375194    2576 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:33:48.375808 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375795    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.375864 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375823    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.375864 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.375837    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.377391 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.377379    2576 policy_none.go:50] "Start"
Nov 04 01:33:48.377428 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.377395    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:33:48.377428 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.377405    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:33:48.379316 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.379306    2576 policy_none.go:44] "Start"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.423252    2576 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427452    2576 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.427485    2576 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427506    2576 server.go:87] "Starting device plugin registration server"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427791    2576 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427801    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427901    2576 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427990    2576 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.427997    2576 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.428605    2576 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.428676    2576 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.436532    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.438097    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.438117    2576 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.438137    2576 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:33:48.439400 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.438169    2576 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 01:33:48.485864 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.485831    2576 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-18.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 01:33:48.485864 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.485837    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-18.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 01:33:48.487104 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.485827    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-18.us-west-1.compute.internal.18e4b39aa30c94c4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-18.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-18.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:33:48.342146244 +0000 UTC m=+0.382074350,LastTimestamp:2026-11-04 01:33:48.342146244 +0000 UTC m=+0.382074350,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-18.us-west-1.compute.internal,}"
Nov 04 01:33:48.528392 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.528324    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.529314 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.529295    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.529391 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.529329    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.529391 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.529342    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.529391 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.529365    2576 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.537003 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.536970    2576 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 01:33:48.537128 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.537048    2576 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 01:33:48.537225 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.537206    2576 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-18.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 01:33:48.538505 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.538489    2576 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"]
Nov 04 01:33:48.538589 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.538557    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.539313 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.539294    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.539369 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.539329    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.539369 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.539343    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.541424 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.541413    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.541604 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.541591    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.541653 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.541638    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.542085 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.542067    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.542159 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.542096    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.542159 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.542108    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.542248 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.542182    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.542248 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.542204    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.542248 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.542216    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.544289 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.544272    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.544340 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.544309    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:48.545365 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.545351    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:48.545447 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.545377    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:48.545447 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.545388    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:48.564393 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.564361    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:33:48.564527 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.564438    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k4knf"
Nov 04 01:33:48.568139 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.568122    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.570950 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.570931    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.648077 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.648052    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:33:48.652296 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.652273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.652395 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.652301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.652395 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.652324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.657050 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.657031    2576 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.657178 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.657059    2576 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-18.us-west-1.compute.internal\": node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:48.752888 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.752849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.752888 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.752886    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.753107 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.752903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.753107 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.752958    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.753107 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.752964    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.753107 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.752967    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.771855 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.771829    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:48.872107 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.872073    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:48.872107 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.872088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.875254 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:48.875234    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:48.973007 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:48.972964    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.073646 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.073600    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.174695 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.174608    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.275407 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.275372    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.280532 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.280510    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 01:33:49.280673 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.280655    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:33:49.280817 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.280726    2576 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aa64b852660404cf9a45925727badae8-e04a1860fe467e96.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-140-18.us-west-1.compute.internal.18e4b39aa4fcd796\": read tcp 10.0.140.18:40642->3.139.49.167:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-140-18.us-west-1.compute.internal.18e4b39aa4fcd796  default   4831 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-18.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-18.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-18.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:33:48 +0000 UTC,LastTimestamp:2026-11-04 01:33:48.539337197 +0000 UTC m=+0.579265301,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-18.us-west-1.compute.internal,}"
Nov 04 01:33:49.376420 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.376372    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.477154 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.477078    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.570027 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.569973    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:28:49 +0000 UTC" deadline="2028-07-21 15:51:59.059341342 +0000 UTC"
Nov 04 01:33:49.570027 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.570014    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15014h18m9.489330071s"
Nov 04 01:33:49.578149 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.578120    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.635499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.635470    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:33:49.636555 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.636538    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:33:49.636680 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.636577    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:33:49.636680 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.636586    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:33:49.679030 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.678995    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.767119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.767051    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:33:49.780122 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.780095    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.880500 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:49.880462    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:33:49.905653 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.905627    2576 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:33:49.934551 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:49.934501    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae6ff4840f4ffd5842f280d0afa74f6.slice/crio-1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13 WatchSource:0}: Error finding container 1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13: Status 404 returned error can't find the container with id 1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13
Nov 04 01:33:49.945338 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.945321    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:33:49.951391 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:49.951370    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:49.955308 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:49.955287    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a45018bbca93ea1080adf030d8fbb7f.slice/crio-4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8 WatchSource:0}: Error finding container 4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8: Status 404 returned error can't find the container with id 4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8
Nov 04 01:33:50.014952 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.014921    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:33:50.017027 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.017011    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:33:50.084275 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.084199    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:33:50.247675 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.247638    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:33:50.315196 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.315169    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:33:50.323894 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.323870    2576 apiserver.go:52] "Watching apiserver"
Nov 04 01:33:50.353266 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.353229    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 01:33:50.437532 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.437490    2576 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:33:50.437855 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.437835    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-zgrx2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7","openshift-image-registry/node-ca-kl2jb","openshift-multus/multus-additional-cni-plugins-vsknk","kube-system/konnectivity-agent-56trq","openshift-cluster-node-tuning-operator/tuned-459wv","openshift-dns/node-resolver-tzccz","openshift-network-diagnostics/network-check-target-68r2l","openshift-network-operator/iptables-alerter-h8x5d","kube-system/global-pull-secret-syncer-p7jkf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-2w6dn","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal","openshift-multus/multus-8lvcd"]
Nov 04 01:33:50.440754 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.440731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.443048 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.443024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:50.443175 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.443103    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:33:50.445354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.445327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.447763 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.447747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.450193 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.450173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.453392 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.453372    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.457842 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.457825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.460266 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.460249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.462732 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.462715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.464260 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-lib-modules\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e13536d8-6703-46a7-9e8e-8136e1ff8979-hosts-file\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.464354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464285    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.464354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464301    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-run\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-tmp\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464618 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7rst\" (UniqueName: \"kubernetes.io/projected/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-kube-api-access-j7rst\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.464618 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464422    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-modprobe-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464618 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-var-lib-kubelet\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464618 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmsgm\" (UniqueName: \"kubernetes.io/projected/c04efb2b-37d6-4ded-a359-ccb48c497b43-kube-api-access-mmsgm\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464618 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-serviceca\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.464817 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cnibin\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.464817 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.464817 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysconfig\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464817 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.464956 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-systemd\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464956 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-sys\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.464956 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-tuned\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.465057 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.464971    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr9hr\" (UniqueName: \"kubernetes.io/projected/e13536d8-6703-46a7-9e8e-8136e1ff8979-kube-api-access-fr9hr\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.465057 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:50.465057 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/016762ac-ad73-46b0-8059-9b4b2b670043-agent-certs\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.465158 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.465158 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-kubernetes\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.465226 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e13536d8-6703-46a7-9e8e-8136e1ff8979-tmp-dir\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.465278 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.465278 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-host\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.465348 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8dml\" (UniqueName: \"kubernetes.io/projected/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-kube-api-access-f8dml\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.465348 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.465300    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:33:50.465434 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-host\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.465434 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-system-cni-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.465538 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/016762ac-ad73-46b0-8059-9b4b2b670043-konnectivity-ca\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.465538 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.465486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-conf\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.467655 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.467620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.469949 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.469935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:50.470014 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.469992    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:33:50.472689 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.472638    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8"}
Nov 04 01:33:50.472753 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.472702    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13"}
Nov 04 01:33:50.498428 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.498406    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.498672 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.498648    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.498753 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.498732    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xwn9r\""
Nov 04 01:33:50.502452 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.502434    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:33:50.502811 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.502798    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zbbgd\""
Nov 04 01:33:50.502862 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.502832    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.502862 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.502847    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.504783 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.504762    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:33:50.505141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.505119    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5jndn\""
Nov 04 01:33:50.505221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.505204    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:33:50.505272 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.505206    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.505272 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.505236    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.505272 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.505239    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:33:50.507344 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.507321    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-sj5vk\""
Nov 04 01:33:50.507489 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.507478    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:33:50.507489 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.507483    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:33:50.510293 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.510275    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.510497 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.510483    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.510699 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.510686    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tfqdd\""
Nov 04 01:33:50.514745 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.514730    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.515189 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.515174    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.515409 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.515395    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:33:50.515482 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.515444    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lhd4j\""
Nov 04 01:33:50.517318 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.517305    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vr5s4\""
Nov 04 01:33:50.549558 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.549528    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:33:50.549810 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.549789    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:33:50.549898 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.549789    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ctj2f\""
Nov 04 01:33:50.549898 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.549844    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.549980 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.549948    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:33:50.550207 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.550188    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.550207 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.550194    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:33:50.550329 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.550197    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:33:50.550329 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.550233    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-shfxh\""
Nov 04 01:33:50.552870 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.552853    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:33:50.552957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.552853    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:33:50.552957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.552860    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:33:50.553278 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.553263    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:33:50.554815 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.554799    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:33:50.554892 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.554843    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hvh55"
Nov 04 01:33:50.566654 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.566771 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-config\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.566771 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-sys\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.566771 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-tuned\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.566771 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-netns\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.566771 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sngl\" (UniqueName: \"kubernetes.io/projected/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kube-api-access-9sngl\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.566957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566772    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-sys\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.566957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-etc-kubernetes\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.566957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9jp\" (UniqueName: \"kubernetes.io/projected/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-kube-api-access-wl9jp\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:50.566957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:50.566957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/016762ac-ad73-46b0-8059-9b4b2b670043-agent-certs\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.566957 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-multus-certs\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566954    2576 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.566998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-etc-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567019    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j4hz\" (UniqueName: \"kubernetes.io/projected/fcb074c6-4262-4601-9730-f4ae23ee938b-kube-api-access-5j4hz\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-socket-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-system-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-k8s-cni-cncf-io\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567225 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6817a7e5-506a-4ae2-8128-5b6601281286-host-slash\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e13536d8-6703-46a7-9e8e-8136e1ff8979-tmp-dir\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-host\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-host\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-slash\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567344    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-host\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-device-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-sys-fs\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcsww\" (UniqueName: \"kubernetes.io/projected/69b134d1-aa78-45f4-a572-7bc4171c5a56-kube-api-access-zcsww\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567445    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-host\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567499 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/016762ac-ad73-46b0-8059-9b4b2b670043-konnectivity-ca\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-conf\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567620    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-systemd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-socket-dir-parent\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-conf-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567675    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-conf\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-run\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-tmp\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-registration-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-kubelet-config\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567798    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-cni-binary-copy\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567853    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-run\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-netns\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.567979 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-multus\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.567997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-netd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568130    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e13536d8-6703-46a7-9e8e-8136e1ff8979-tmp-dir\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-dbus\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-kubelet\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq6zm\" (UniqueName: \"kubernetes.io/projected/6817a7e5-506a-4ae2-8128-5b6601281286-kube-api-access-zq6zm\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568275    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568302    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cnibin\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568280    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cnibin\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568335    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-log-socket\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-bin\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcb074c6-4262-4601-9730-f4ae23ee938b-ovn-node-metrics-cert\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-systemd\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-env-overrides\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.568760 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568545    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-systemd\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568553    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-bin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr9hr\" (UniqueName: \"kubernetes.io/projected/e13536d8-6703-46a7-9e8e-8136e1ff8979-kube-api-access-fr9hr\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568695    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/016762ac-ad73-46b0-8059-9b4b2b670043-konnectivity-ca\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-hostroot\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-kubernetes\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-ovn\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568898    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568910    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-kubernetes\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-cnibin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.568952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8dml\" (UniqueName: \"kubernetes.io/projected/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-kube-api-access-f8dml\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-node-log\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-script-lib\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-system-cni-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-lib-modules\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.569571 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569115    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-system-cni-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-kubelet\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-daemon-config\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569223    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-lib-modules\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e13536d8-6703-46a7-9e8e-8136e1ff8979-hosts-file\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569290    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-var-lib-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569306    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e13536d8-6703-46a7-9e8e-8136e1ff8979-hosts-file\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569343    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rst\" (UniqueName: \"kubernetes.io/projected/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-kube-api-access-j7rst\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-modprobe-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569436    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-var-lib-kubelet\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569456    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-modprobe-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569474    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-var-lib-kubelet\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570119 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmsgm\" (UniqueName: \"kubernetes.io/projected/c04efb2b-37d6-4ded-a359-ccb48c497b43-kube-api-access-mmsgm\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-systemd-units\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6817a7e5-506a-4ae2-8128-5b6601281286-iptables-alerter-script\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-serviceca\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysconfig\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569710    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysconfig\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.569953    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-serviceca\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.570605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.570552    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-tuned\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570823 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.570705    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-tmp\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.570958 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.570941    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/016762ac-ad73-46b0-8059-9b4b2b670043-agent-certs\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.631529 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.631434    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:33:50.631529 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.631466    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:33:50.631529 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.631480    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fq5xp for pod openshift-network-diagnostics/network-check-target-68r2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:50.631764 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.631562    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp podName:a5bea9b9-e510-4b8b-bc95-5ad38581b40c nodeName:}" failed. No retries permitted until 2026-11-04 01:33:51.131538667 +0000 UTC m=+3.171466779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fq5xp" (UniqueName: "kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp") pod "network-check-target-68r2l" (UID: "a5bea9b9-e510-4b8b-bc95-5ad38581b40c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:50.634994 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.634954    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr9hr\" (UniqueName: \"kubernetes.io/projected/e13536d8-6703-46a7-9e8e-8136e1ff8979-kube-api-access-fr9hr\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.635141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.635025    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8dml\" (UniqueName: \"kubernetes.io/projected/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-kube-api-access-f8dml\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.635386 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.635366    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rst\" (UniqueName: \"kubernetes.io/projected/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-kube-api-access-j7rst\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.635529 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.635501    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmsgm\" (UniqueName: \"kubernetes.io/projected/c04efb2b-37d6-4ded-a359-ccb48c497b43-kube-api-access-mmsgm\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.670841 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.670800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-multus-certs\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.670954 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.670859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-etc-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.670954 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.670877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j4hz\" (UniqueName: \"kubernetes.io/projected/fcb074c6-4262-4601-9730-f4ae23ee938b-kube-api-access-5j4hz\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.670954 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.670882    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-multus-certs\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.670954 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.670917    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-etc-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.670970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-socket-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-system-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-k8s-cni-cncf-io\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6817a7e5-506a-4ae2-8128-5b6601281286-host-slash\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671071    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-system-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671078    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-k8s-cni-cncf-io\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-slash\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671107    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-socket-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671117    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671130    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-slash\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671181 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671140    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-device-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-sys-fs\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671117    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6817a7e5-506a-4ae2-8128-5b6601281286-host-slash\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671229    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-device-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsww\" (UniqueName: \"kubernetes.io/projected/69b134d1-aa78-45f4-a572-7bc4171c5a56-kube-api-access-zcsww\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671266    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-sys-fs\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-systemd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-socket-dir-parent\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-conf-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671344    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-systemd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671375    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671385    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-socket-dir-parent\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-registration-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671424    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-conf-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671447    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-registration-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.671695 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-kubelet-config\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-cni-binary-copy\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-netns\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-multus\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-netd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-dbus\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671549    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-netns\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-kubelet\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6zm\" (UniqueName: \"kubernetes.io/projected/6817a7e5-506a-4ae2-8128-5b6601281286-kube-api-access-zq6zm\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671656    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-netd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671614    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-kubelet-config\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-log-socket\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-bin\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671737    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-kubelet\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671796    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-log-socket\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671798    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-dbus\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671844    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-bin\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcb074c6-4262-4601-9730-f4ae23ee938b-ovn-node-metrics-cert\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.672495 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671824    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-multus\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-env-overrides\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-bin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.671969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672023    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-bin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672054    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-hostroot\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-ovn\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672101    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-cnibin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672133    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-hostroot\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-node-log\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672160    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-ovn\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672172    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-node-log\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.672175    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672180    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-cnibin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.673564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-script-lib\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.672259    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret podName:76b11319-689c-4ba5-83ec-afe254b80174 nodeName:}" failed. No retries permitted until 2026-11-04 01:33:51.172233817 +0000 UTC m=+3.212161932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret") pod "global-pull-secret-syncer-p7jkf" (UID: "76b11319-689c-4ba5-83ec-afe254b80174") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672405    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-env-overrides\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672410    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-cni-binary-copy\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-kubelet\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672831    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-kubelet\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-daemon-config\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672915    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-var-lib-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-systemd-units\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672987    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-var-lib-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6817a7e5-506a-4ae2-8128-5b6601281286-iptables-alerter-script\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.672999    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-script-lib\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673094    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-systemd-units\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.674486 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673123    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.673100    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-config\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:50.673220    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:33:51.173184957 +0000 UTC m=+3.213113069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-netns\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sngl\" (UniqueName: \"kubernetes.io/projected/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kube-api-access-9sngl\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-etc-kubernetes\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9jp\" (UniqueName: \"kubernetes.io/projected/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-kube-api-access-wl9jp\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673420    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-daemon-config\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673468    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-netns\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673504    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6817a7e5-506a-4ae2-8128-5b6601281286-iptables-alerter-script\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673505    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-etc-kubernetes\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.673634    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-config\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.675071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.674787    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcb074c6-4262-4601-9730-f4ae23ee938b-ovn-node-metrics-cert\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.739209 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.739169    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6zm\" (UniqueName: \"kubernetes.io/projected/6817a7e5-506a-4ae2-8128-5b6601281286-kube-api-access-zq6zm\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.739377 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.739215    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j4hz\" (UniqueName: \"kubernetes.io/projected/fcb074c6-4262-4601-9730-f4ae23ee938b-kube-api-access-5j4hz\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.739377 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.739215    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcsww\" (UniqueName: \"kubernetes.io/projected/69b134d1-aa78-45f4-a572-7bc4171c5a56-kube-api-access-zcsww\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.739510 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.739464    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sngl\" (UniqueName: \"kubernetes.io/projected/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kube-api-access-9sngl\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.739573 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.739556    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9jp\" (UniqueName: \"kubernetes.io/projected/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-kube-api-access-wl9jp\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:50.752559 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.752537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:33:50.759554 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.759529    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:33:50.765386 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.765355    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13536d8_6703_46a7_9e8e_8136e1ff8979.slice/crio-54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b WatchSource:0}: Error finding container 54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b: Status 404 returned error can't find the container with id 54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b
Nov 04 01:33:50.769099 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.769071    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3ebfff_ac2e_4429_ac1e_1fb87d7599d0.slice/crio-483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e WatchSource:0}: Error finding container 483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e: Status 404 returned error can't find the container with id 483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e
Nov 04 01:33:50.777669 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.777641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:50.778763 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.778746    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:33:50.783541 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.783518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:33:50.786094 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.786075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:33:50.799967 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.799879    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbd09a6_31fc_4e11_bf84_da0864ec84ff.slice/crio-dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d WatchSource:0}: Error finding container dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d: Status 404 returned error can't find the container with id dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d
Nov 04 01:33:50.803764 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.803735    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6817a7e5_506a_4ae2_8128_5b6601281286.slice/crio-53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298 WatchSource:0}: Error finding container 53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298: Status 404 returned error can't find the container with id 53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298
Nov 04 01:33:50.808438 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.808417    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lvcd"
Nov 04 01:33:50.810677 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.810656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:33:50.814547 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:50.814523    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:33:50.820172 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.820150    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b134d1_aa78_45f4_a572_7bc4171c5a56.slice/crio-2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109 WatchSource:0}: Error finding container 2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109: Status 404 returned error can't find the container with id 2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109
Nov 04 01:33:50.824662 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.824629    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65a7a37_b3e1_46a1_9ed7_204de523fdc7.slice/crio-8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b WatchSource:0}: Error finding container 8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b: Status 404 returned error can't find the container with id 8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b
Nov 04 01:33:50.825585 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:33:50.825564    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb074c6_4262_4601_9730_f4ae23ee938b.slice/crio-ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad WatchSource:0}: Error finding container ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad: Status 404 returned error can't find the container with id ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad
Nov 04 01:33:51.176751 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.176709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:51.176960 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.176780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:51.176960 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.176820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:51.176960 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.176905    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:51.176960 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.176948    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:51.177158 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.176968    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:33:51.177158 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.176984    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:33:51.177158 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.176997    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fq5xp for pod openshift-network-diagnostics/network-check-target-68r2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:51.177158 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.176996    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret podName:76b11319-689c-4ba5-83ec-afe254b80174 nodeName:}" failed. No retries permitted until 2026-11-04 01:33:52.176972909 +0000 UTC m=+4.216901011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret") pod "global-pull-secret-syncer-p7jkf" (UID: "76b11319-689c-4ba5-83ec-afe254b80174") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:51.177158 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.177057    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:33:52.177040784 +0000 UTC m=+4.216968882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:51.177158 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:51.177074    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp podName:a5bea9b9-e510-4b8b-bc95-5ad38581b40c nodeName:}" failed. No retries permitted until 2026-11-04 01:33:52.177063185 +0000 UTC m=+4.216991284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fq5xp" (UniqueName: "kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp") pod "network-check-target-68r2l" (UID: "a5bea9b9-e510-4b8b-bc95-5ad38581b40c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:51.447865 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.447694    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7" event={"ID":"c65a7a37-b3e1-46a1-9ed7-204de523fdc7","Type":"ContainerStarted","Data":"8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b"}
Nov 04 01:33:51.449279 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.449249    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8x5d" event={"ID":"6817a7e5-506a-4ae2-8128-5b6601281286","Type":"ContainerStarted","Data":"53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298"}
Nov 04 01:33:51.450712 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.450683    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-459wv" event={"ID":"c04efb2b-37d6-4ded-a359-ccb48c497b43","Type":"ContainerStarted","Data":"3a054ccaf4d5579e9451cab9ad576b23f6fc4755e146ab38aeb00df964a080a2"}
Nov 04 01:33:51.452223 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.452195    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzccz" event={"ID":"e13536d8-6703-46a7-9e8e-8136e1ff8979","Type":"ContainerStarted","Data":"54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b"}
Nov 04 01:33:51.454086 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.454059    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad"}
Nov 04 01:33:51.455550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.455526    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lvcd" event={"ID":"69b134d1-aa78-45f4-a572-7bc4171c5a56","Type":"ContainerStarted","Data":"2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109"}
Nov 04 01:33:51.456814 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.456790    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerStarted","Data":"dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d"}
Nov 04 01:33:51.458297 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.458265    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-56trq" event={"ID":"016762ac-ad73-46b0-8059-9b4b2b670043","Type":"ContainerStarted","Data":"1005ba1b09ed7ea8bf61b984c4badf72a4abb1abfac03100104c15d6e121bd27"}
Nov 04 01:33:51.460332 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.460306    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kl2jb" event={"ID":"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0","Type":"ContainerStarted","Data":"483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e"}
Nov 04 01:33:51.558484 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.558445    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:28:50 +0000 UTC" deadline="2028-06-23 03:55:16.556003529 +0000 UTC"
Nov 04 01:33:51.558484 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:51.558481    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14330h21m24.99752685s"
Nov 04 01:33:52.184564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.184465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:52.184564 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.184541    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:52.184808 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.184588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:52.184808 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.184777    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:33:52.184808 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.184797    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:33:52.184808 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.184811    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fq5xp for pod openshift-network-diagnostics/network-check-target-68r2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:52.185032 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.184874    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp podName:a5bea9b9-e510-4b8b-bc95-5ad38581b40c nodeName:}" failed. No retries permitted until 2026-11-04 01:33:54.184856052 +0000 UTC m=+6.224784160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fq5xp" (UniqueName: "kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp") pod "network-check-target-68r2l" (UID: "a5bea9b9-e510-4b8b-bc95-5ad38581b40c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:52.185032 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.184947    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:52.185032 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.184979    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret podName:76b11319-689c-4ba5-83ec-afe254b80174 nodeName:}" failed. No retries permitted until 2026-11-04 01:33:54.18496942 +0000 UTC m=+6.224897539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret") pod "global-pull-secret-syncer-p7jkf" (UID: "76b11319-689c-4ba5-83ec-afe254b80174") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:52.185190 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.185034    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:52.185190 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.185067    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:33:54.185054887 +0000 UTC m=+6.224982985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:52.441646 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.441568    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:52.441828 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.441712    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:33:52.441828 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.441800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:52.441950 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.441868    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:33:52.441950 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.441916    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:52.442061 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:52.441965    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:33:52.559358 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.559291    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:28:50 +0000 UTC" deadline="2028-07-04 09:34:43.581464047 +0000 UTC"
Nov 04 01:33:52.559358 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:52.559355    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14600h0m51.022112789s"
Nov 04 01:33:53.487638 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:53.487306    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"547e2cede1fc8a4092d19734f469e16a00b49ff92f8a8f0d3ba0470812d2c602"}
Nov 04 01:33:53.631099 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:53.631027    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" podStartSLOduration=3.631006369 podStartE2EDuration="3.631006369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:33:53.630044477 +0000 UTC m=+5.669972595" watchObservedRunningTime="2026-11-04 01:33:53.631006369 +0000 UTC m=+5.670934488"
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.207180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.207240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.207302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207436    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207500    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret podName:76b11319-689c-4ba5-83ec-afe254b80174 nodeName:}" failed. No retries permitted until 2026-11-04 01:33:58.207480442 +0000 UTC m=+10.247408556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret") pod "global-pull-secret-syncer-p7jkf" (UID: "76b11319-689c-4ba5-83ec-afe254b80174") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207569    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207608    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:33:58.207592036 +0000 UTC m=+10.247520132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207733    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207754    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207768    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fq5xp for pod openshift-network-diagnostics/network-check-target-68r2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:54.207844 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.207811    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp podName:a5bea9b9-e510-4b8b-bc95-5ad38581b40c nodeName:}" failed. No retries permitted until 2026-11-04 01:33:58.207798449 +0000 UTC m=+10.247726545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fq5xp" (UniqueName: "kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp") pod "network-check-target-68r2l" (UID: "a5bea9b9-e510-4b8b-bc95-5ad38581b40c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:54.440785 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.440757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:54.440921 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.440895    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:33:54.440991 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.440968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:54.441299 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.441059    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:33:54.441299 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.441125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:54.441299 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:54.441199    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:33:54.496691 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.495749    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerStarted","Data":"04bebed6bb7f38f5400f0da91809b3c4fbf6f5f6569e20d95ed9309cbc35b314"}
Nov 04 01:33:54.508139 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.506844    2576 generic.go:356] "Generic (PLEG): container finished" podID="9a45018bbca93ea1080adf030d8fbb7f" containerID="3524f371f99d71361dae2e005d3da2297a709c26655dc3970b1351b353d090be" exitCode=0
Nov 04 01:33:54.508139 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:54.507885    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerDied","Data":"3524f371f99d71361dae2e005d3da2297a709c26655dc3970b1351b353d090be"}
Nov 04 01:33:55.517887 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:55.517844    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"541e7628a44bd14f5c6718b103113b2ad3ea2bc3e946b1deab2667ec402dbbee"}
Nov 04 01:33:55.522721 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:55.522680    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="04bebed6bb7f38f5400f0da91809b3c4fbf6f5f6569e20d95ed9309cbc35b314" exitCode=0
Nov 04 01:33:55.522902 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:55.522746    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerDied","Data":"04bebed6bb7f38f5400f0da91809b3c4fbf6f5f6569e20d95ed9309cbc35b314"}
Nov 04 01:33:55.779588 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:55.779473    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" podStartSLOduration=5.779456892 podStartE2EDuration="5.779456892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:33:55.64653871 +0000 UTC m=+7.686466829" watchObservedRunningTime="2026-11-04 01:33:55.779456892 +0000 UTC m=+7.819385008"
Nov 04 01:33:56.441010 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:56.440978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:56.441205 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:56.441115    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:33:56.441265 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:56.441209    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:56.441322 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:56.441297    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:33:56.441377 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:56.441358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:56.441527 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:56.441430    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:33:56.530963 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:56.530888    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-56trq" event={"ID":"016762ac-ad73-46b0-8059-9b4b2b670043","Type":"ContainerStarted","Data":"fc51a36f58f6a4150266cd0114ddeda966bc4d9903900432af68f44b852d44ab"}
Nov 04 01:33:57.538780 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:57.538742    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kl2jb" event={"ID":"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0","Type":"ContainerStarted","Data":"aa6b447c95c4f310492d687362a440f3a86d0624ba46331806ad6f9169fa8548"}
Nov 04 01:33:57.543289 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:57.543261    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7" event={"ID":"c65a7a37-b3e1-46a1-9ed7-204de523fdc7","Type":"ContainerStarted","Data":"bc7ffe3597e96c244a1d1c5e49d14a8785d7feace14811a44d7b3c243ddffe77"}
Nov 04 01:33:57.655404 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:57.655353    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kl2jb" podStartSLOduration=2.759146818 podStartE2EDuration="8.655334953s" totalImagesPullingTime="5.896188135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:33:57.655001635 +0000 UTC m=+9.694929753" watchObservedRunningTime="2026-11-04 01:33:57.655334953 +0000 UTC m=+9.695263070"
Nov 04 01:33:57.655624 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:57.655548    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-56trq" podStartSLOduration=3.625256682 podStartE2EDuration="8.655540457s" totalImagesPullingTime="5.030283775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:33:56.657797154 +0000 UTC m=+8.697725272" watchObservedRunningTime="2026-11-04 01:33:57.655540457 +0000 UTC m=+9.695468572"
Nov 04 01:33:58.242725 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:58.242686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:58.242949 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:58.242746    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:58.242949 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:58.242771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:58.242949 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.242877    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:58.242949 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.242928    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:06.242915792 +0000 UTC m=+18.282843885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:33:58.243244 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.242994    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:33:58.243244 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.243003    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:33:58.243244 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.243011    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fq5xp for pod openshift-network-diagnostics/network-check-target-68r2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:58.243244 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.243039    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp podName:a5bea9b9-e510-4b8b-bc95-5ad38581b40c nodeName:}" failed. No retries permitted until 2026-11-04 01:34:06.243031216 +0000 UTC m=+18.282959317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fq5xp" (UniqueName: "kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp") pod "network-check-target-68r2l" (UID: "a5bea9b9-e510-4b8b-bc95-5ad38581b40c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:33:58.243244 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.243083    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:58.243244 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.243113    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret podName:76b11319-689c-4ba5-83ec-afe254b80174 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:06.243103513 +0000 UTC m=+18.283031619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret") pod "global-pull-secret-syncer-p7jkf" (UID: "76b11319-689c-4ba5-83ec-afe254b80174") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:33:58.441591 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:58.440925    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:33:58.441591 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.441056    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:33:58.441591 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:58.441144    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:33:58.441591 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.441239    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:33:58.441591 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:58.441298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:33:58.441591 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:33:58.441358    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:33:59.627741 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:59.625891    2576 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:33:59.879180 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:59.878551    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:33:59.879585 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:33:59.879552    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:00.441002 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.440738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:00.441166 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:00.441139    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:00.441273 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.440779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:00.441515 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:00.441430    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:00.441583 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.441527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:00.441874 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:00.441856    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:00.444826 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.444063    2576 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:33:59.625912354Z","UUID":"419236e2-d9ee-41a9-8821-1dff4a63da2a","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:34:00.445838 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.445817    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:34:00.445938 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.445850    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:34:00.556820 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.556775    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7" event={"ID":"c65a7a37-b3e1-46a1-9ed7-204de523fdc7","Type":"ContainerStarted","Data":"b2a4edaf9b64c2b19da31897bd320b62becd9ba9fe609a48dff2062f501c0180"}
Nov 04 01:34:00.558730 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.558701    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzccz" event={"ID":"e13536d8-6703-46a7-9e8e-8136e1ff8979","Type":"ContainerStarted","Data":"9b8171ca3bddb35c970efead0b3d761ae19e7a0b3fa98927275f3703627cde05"}
Nov 04 01:34:00.560474 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.560445    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lvcd" event={"ID":"69b134d1-aa78-45f4-a572-7bc4171c5a56","Type":"ContainerStarted","Data":"c3c17812f9d73ea6b8d25c1270599b9bc6f47e07baa6cccd35306efdde6ec9d4"}
Nov 04 01:34:00.563478 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.563452    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="091b3e2f38d2b9fb179cc887af7fa4027cef193102a14b109a81e0594e1e520b" exitCode=0
Nov 04 01:34:00.564352 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.564326    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerDied","Data":"091b3e2f38d2b9fb179cc887af7fa4027cef193102a14b109a81e0594e1e520b"}
Nov 04 01:34:00.796000 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.795941    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-8lvcd" podStartSLOduration=3.186202332 podStartE2EDuration="11.795922502s" totalImagesPullingTime="8.60972017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:00.795774156 +0000 UTC m=+12.835702269" watchObservedRunningTime="2026-11-04 01:34:00.795922502 +0000 UTC m=+12.835850620"
Nov 04 01:34:00.796786 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:00.796744    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-tzccz" podStartSLOduration=3.019378819 podStartE2EDuration="11.796728725s" totalImagesPullingTime="8.777349906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:00.675839485 +0000 UTC m=+12.715767598" watchObservedRunningTime="2026-11-04 01:34:00.796728725 +0000 UTC m=+12.836656843"
Nov 04 01:34:01.568777 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:01.568703    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7" event={"ID":"c65a7a37-b3e1-46a1-9ed7-204de523fdc7","Type":"ContainerStarted","Data":"4b6390f859d093c13ceb82dce9c43e93613dae9dd0c05f3dfbe68df5f817a3ac"}
Nov 04 01:34:01.570475 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:01.570436    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8x5d" event={"ID":"6817a7e5-506a-4ae2-8128-5b6601281286","Type":"ContainerStarted","Data":"ffc92fb36d3617923148f06bb6738f5bc48af2cbaa612f8fd5d964bfbc2af338"}
Nov 04 01:34:01.572629 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:01.572602    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-459wv" event={"ID":"c04efb2b-37d6-4ded-a359-ccb48c497b43","Type":"ContainerStarted","Data":"35735fa3040248e8e236f7298b2f2d8c756a62d8fb15ed5fdea9a5fb21a07fef"}
Nov 04 01:34:01.808413 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:01.808366    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-459wv" podStartSLOduration=2.287620832 podStartE2EDuration="12.808351958s" totalImagesPullingTime="10.520731126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:01.808312393 +0000 UTC m=+13.848240508" watchObservedRunningTime="2026-11-04 01:34:01.808351958 +0000 UTC m=+13.848280071"
Nov 04 01:34:01.809103 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:01.809070    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7" podStartSLOduration=2.972368016 podStartE2EDuration="12.809058858s" totalImagesPullingTime="9.836690842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:01.690137602 +0000 UTC m=+13.730065717" watchObservedRunningTime="2026-11-04 01:34:01.809058858 +0000 UTC m=+13.848986974"
Nov 04 01:34:01.923965 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:01.923896    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h8x5d" podStartSLOduration=3.984037881 podStartE2EDuration="12.923877253s" totalImagesPullingTime="8.939839372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:01.923653761 +0000 UTC m=+13.963581874" watchObservedRunningTime="2026-11-04 01:34:01.923877253 +0000 UTC m=+13.963805366"
Nov 04 01:34:02.439447 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:02.439358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:02.439447 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:02.439402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:02.439718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:02.439370    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:02.439718 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:02.439489    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:02.439718 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:02.439563    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:02.439718 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:02.439644    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:02.577271 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:02.577236    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="d964877aae60435648b96594342e62f9ec630a12e36d8809cc7f514570d9479c" exitCode=0
Nov 04 01:34:02.578118 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:02.578098    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerDied","Data":"d964877aae60435648b96594342e62f9ec630a12e36d8809cc7f514570d9479c"}
Nov 04 01:34:04.438592 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:04.438545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:04.439301 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:04.438597    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:04.439301 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:04.438550    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:04.439301 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:04.438712    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:04.439301 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:04.438771    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:04.440601 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:04.439850    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:04.584853 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:04.584820    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="e53cf97be7611f1908941a1102b4a3380e398f29c2d03716689b00fc4fbfaf2d" exitCode=0
Nov 04 01:34:04.585012 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:04.584867    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerDied","Data":"e53cf97be7611f1908941a1102b4a3380e398f29c2d03716689b00fc4fbfaf2d"}
Nov 04 01:34:05.589927 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:05.589702    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"ad163c4b9815bb26e092be9421dc8aafedfebb971b2e7c9db103b11aeca7741e"}
Nov 04 01:34:05.590344 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:05.589990    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"72b11d786d4ffd460670840123f0d52c0a7283e23c40c37b4d71527e88a13405"}
Nov 04 01:34:05.590344 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:05.590040    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"13f135f666926ff47ec8385b4363aa4c5128952a85a896980bc584efe49b1fd0"}
Nov 04 01:34:05.590344 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:05.590083    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"f5094519e103c04f739355996ffdd4dfc13eac52da3a4a726fabc654dc1853d4"}
Nov 04 01:34:06.299802 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.299563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.299848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.299746    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.299888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.299916    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.299944    2576 projected.go:196] Error preparing data for projected volume kube-api-access-fq5xp for pod openshift-network-diagnostics/network-check-target-68r2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.299967    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:06.299981 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.299979    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:06.300209 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.299998    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp podName:a5bea9b9-e510-4b8b-bc95-5ad38581b40c nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.299981629 +0000 UTC m=+34.339909741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fq5xp" (UniqueName: "kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp") pod "network-check-target-68r2l" (UID: "a5bea9b9-e510-4b8b-bc95-5ad38581b40c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:06.300209 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.300017    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret podName:76b11319-689c-4ba5-83ec-afe254b80174 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.300008186 +0000 UTC m=+34.339936279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret") pod "global-pull-secret-syncer-p7jkf" (UID: "76b11319-689c-4ba5-83ec-afe254b80174") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:06.300209 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.300032    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.300026281 +0000 UTC m=+34.339954374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:06.438848 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.438819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:06.439003 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.438890    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:06.439003 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.438882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:06.439111 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.438996    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:06.439111 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.439059    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:06.439111 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:06.439091    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:06.595095 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.595068    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"4eae09eda9cc6bf288178274288f840c2019312fc102050a338c3327be2ae99c"}
Nov 04 01:34:06.595451 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:06.595135    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"059ad450b83a3b2035b4a12dd9d04b3a7d1da6a157f5cf40239db53535121014"}
Nov 04 01:34:07.599928 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:07.599895    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="5a8e1765847b8d10eb3e1926e6965bbdd3c2eac7d798a8d8b998bdfba0107497" exitCode=0
Nov 04 01:34:07.600366 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:07.599939    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerDied","Data":"5a8e1765847b8d10eb3e1926e6965bbdd3c2eac7d798a8d8b998bdfba0107497"}
Nov 04 01:34:08.439205 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:08.439175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:08.439363 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:08.439261    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:08.439363 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:08.439287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:08.439363 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:08.439287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:08.439534 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:08.439359    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:08.439534 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:08.439475    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:08.605334 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:08.605301    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"3d32954d35a49fcd2c69d3dd062fd15553d6e4ac0daf78e2782151fc96686a25"}
Nov 04 01:34:08.608196 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:08.608167    2576 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="deddbd24546f687ca9eb366a921520e5f02bcf6beb6be7d591802599355a920f" exitCode=0
Nov 04 01:34:08.608301 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:08.608215    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerDied","Data":"deddbd24546f687ca9eb366a921520e5f02bcf6beb6be7d591802599355a920f"}
Nov 04 01:34:09.435658 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:09.435627    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:09.435922 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:09.435757    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:09.436324 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:09.436299    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:09.613800 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:09.613771    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsknk" event={"ID":"cfbd09a6-31fc-4e11-bf84-da0864ec84ff","Type":"ContainerStarted","Data":"555ecd4f26ebb5c2fa38e861f5ef00d45f4ad22f9dda631859c074e00ead8e56"}
Nov 04 01:34:09.737282 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:09.737197    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vsknk" podStartSLOduration=8.645012974 podStartE2EDuration="20.737184703s" totalImagesPullingTime="12.092171729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:09.736820672 +0000 UTC m=+21.776748787" watchObservedRunningTime="2026-11-04 01:34:09.737184703 +0000 UTC m=+21.777112817"
Nov 04 01:34:10.441321 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:10.441125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:10.441507 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:10.441150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:10.441507 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:10.441403    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:10.441507 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:10.441177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:10.441507 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:10.441466    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:10.441718 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:10.441547    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:10.619701 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:10.619670    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" event={"ID":"fcb074c6-4262-4601-9730-f4ae23ee938b","Type":"ContainerStarted","Data":"c39400455b2eefcf88a3fc2bf4b0f9dd36d35cfdc474b53c0134b21dde49f4c4"}
Nov 04 01:34:10.800209 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:10.800114    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" podStartSLOduration=7.276827761 podStartE2EDuration="21.800102097s" totalImagesPullingTime="14.523274336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:10.799774507 +0000 UTC m=+22.839702620" watchObservedRunningTime="2026-11-04 01:34:10.800102097 +0000 UTC m=+22.840030211"
Nov 04 01:34:11.622532 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:11.622501    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:11.623274 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:11.623246    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:11.623478 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:11.623462    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:11.640244 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:11.640222    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:11.640350 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:11.640320    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:12.438522 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.438489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:12.438522 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.438513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:12.438797 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.438641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:12.438797 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:12.438640    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:12.438797 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:12.438706    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:12.438797 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:12.438780    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:12.624518 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.624495    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:12.703435 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.703352    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p7jkf"]
Nov 04 01:34:12.703612 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.703459    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:12.703612 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:12.703554    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:12.706125 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.706102    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-68r2l"]
Nov 04 01:34:12.706241 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.706181    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:12.706312 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:12.706264    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:12.709260 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.709241    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2w6dn"]
Nov 04 01:34:12.709343 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:12.709331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:12.709459 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:12.709440    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:13.551113 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:13.551079    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:14.439453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:14.439421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:14.439925 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:14.439421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:14.439925 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:14.439546    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:14.439925 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:14.439528    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:14.439925 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:14.439634    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:14.439925 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:14.439709    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:14.643427 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:14.643383    2576 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2" podUID="fcb074c6-4262-4601-9730-f4ae23ee938b" containerName="ovnkube-controller" probeResult="failure" output=""
Nov 04 01:34:16.441360 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:16.441331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:16.441819 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:16.441441    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7jkf" podUID="76b11319-689c-4ba5-83ec-afe254b80174"
Nov 04 01:34:16.441819 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:16.441445    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:16.441819 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:16.441509    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-68r2l" podUID="a5bea9b9-e510-4b8b-bc95-5ad38581b40c"
Nov 04 01:34:16.441819 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:16.441445    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:16.441819 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:16.441594    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2w6dn" podUID="10bc58fa-af70-41c2-9e8c-0cf7774b67fe"
Nov 04 01:34:18.242088 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.241896    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:34:18.242545 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.242239    2576 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:34:18.358767 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.358736    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-f66h9"]
Nov 04 01:34:18.376168 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.376139    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-j4l4l"]
Nov 04 01:34:18.376308 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.376260    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:18.392714 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.392695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:18.392831 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.392696    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-557fcb4957-k6rbt"]
Nov 04 01:34:18.413102 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.413082    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-sx466"]
Nov 04 01:34:18.413227 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.413210    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.420985 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.420966    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"]
Nov 04 01:34:18.421084 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.421069    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:18.422800 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.422781    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d844f87d4-6hn9v"]
Nov 04 01:34:18.422906 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.422890    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.424713 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.424677    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"]
Nov 04 01:34:18.424789 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.424777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.426022 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.426006    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54848578cf-gq6dc"]
Nov 04 01:34:18.426144 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.426129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.427555 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.427529    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"]
Nov 04 01:34:18.427743 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.427687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.430145 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.430128    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"]
Nov 04 01:34:18.431050 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.430783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.432341 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.432325    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8zdbz"]
Nov 04 01:34:18.432447 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.432431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:18.433756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.433741    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"]
Nov 04 01:34:18.433839 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.433805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.435134 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.435114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.438322 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.438302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:18.438417 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.438363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:18.438417 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.438376    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:18.461382 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.461360    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.461956 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.461940    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.462093 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.462080    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"]
Nov 04 01:34:18.464556 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.464538    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"]
Nov 04 01:34:18.464683 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.464664    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:18.466333 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.466316    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"]
Nov 04 01:34:18.466819 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.466799    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 01:34:18.466909 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.466831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.467121 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.467099    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 01:34:18.467268 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.467247    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 01:34:18.467354 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.467341    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jhtpj\""
Nov 04 01:34:18.468671 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.468641    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jn9bb"]
Nov 04 01:34:18.468762 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.468728    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:18.470515 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.470491    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m66qn"]
Nov 04 01:34:18.470641 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.470626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.472640 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.472620    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-j4l4l"]
Nov 04 01:34:18.472734 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.472658    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-f66h9"]
Nov 04 01:34:18.472734 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.472679    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-sx466"]
Nov 04 01:34:18.472734 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.472699    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.473783 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.472702    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"]
Nov 04 01:34:18.475487 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.475469    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557fcb4957-k6rbt"]
Nov 04 01:34:18.479395 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.479370    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d844f87d4-6hn9v"]
Nov 04 01:34:18.486738 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.486844 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp9mj\" (UniqueName: \"kubernetes.io/projected/c7411ec4-7d75-44ad-ae7c-fae47eee36f4-kube-api-access-qp9mj\") pod \"volume-data-source-validator-58c9f5f46d-4w7fx\" (UID: \"c7411ec4-7d75-44ad-ae7c-fae47eee36f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:18.486844 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-default-certificate\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.486844 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.487015 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.487015 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486897    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8zdbz"]
Nov 04 01:34:18.487015 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.487015 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.486963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8ec90134-2d9f-418f-98e5-877305e1a0b2-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.487177 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.487177 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.487268 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47p5l\" (UniqueName: \"kubernetes.io/projected/928e6807-f385-42a2-89d5-e9ed3b1089cf-kube-api-access-47p5l\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.487324 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.487387 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.487437 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-snapshots\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.487492 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp9v2\" (UniqueName: \"kubernetes.io/projected/9377de09-4111-4499-a1d7-62e407f76422-kube-api-access-hp9v2\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.487605 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mc9h\" (UniqueName: \"kubernetes.io/projected/081f052e-3e74-4fd5-8d12-d91aa017a999-kube-api-access-6mc9h\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.487667 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.487719 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df3c83fd-42b4-423d-a2a2-9da642a21118-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.487764 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.487748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.488118 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9km5\" (UniqueName: \"kubernetes.io/projected/8ec90134-2d9f-418f-98e5-877305e1a0b2-kube-api-access-q9km5\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.488173 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.488211 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr76c\" (UniqueName: \"kubernetes.io/projected/df3c83fd-42b4-423d-a2a2-9da642a21118-kube-api-access-nr76c\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.488286 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.488335 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.488385 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/081f052e-3e74-4fd5-8d12-d91aa017a999-config\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.488488 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.488488 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.488488 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67jjd\" (UniqueName: \"kubernetes.io/projected/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db-kube-api-access-67jjd\") pod \"kas-connection-checker-685895c7c9-sx466\" (UID: \"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db\") " pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:18.488636 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvxs\" (UniqueName: \"kubernetes.io/projected/81cc5d87-5a2f-43e5-ba84-04993c15476e-kube-api-access-2qvxs\") pod \"kas-connection-checker-685895c7c9-j4l4l\" (UID: \"81cc5d87-5a2f-43e5-ba84-04993c15476e\") " pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:18.488636 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.488636 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.488747 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df3c83fd-42b4-423d-a2a2-9da642a21118-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.488808 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4d5f\" (UniqueName: \"kubernetes.io/projected/6a1464fb-a344-4bd9-8406-6fe464ff4fa0-kube-api-access-l4d5f\") pod \"kas-connection-checker-685895c7c9-f66h9\" (UID: \"6a1464fb-a344-4bd9-8406-6fe464ff4fa0\") " pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:18.488859 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-service-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.488907 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.488976 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.488959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-tmp\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.489044 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489025    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.489097 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.489149 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-stats-auth\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.489202 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.489202 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pzgb\" (UniqueName: \"kubernetes.io/projected/07ebf665-177c-48e7-969c-28057a4ff1a4-kube-api-access-4pzgb\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.489284 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9377de09-4111-4499-a1d7-62e407f76422-serving-cert\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.489326 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/081f052e-3e74-4fd5-8d12-d91aa017a999-serving-cert\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.489382 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.489361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.492647 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.492626    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"]
Nov 04 01:34:18.511504 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.511481    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"]
Nov 04 01:34:18.523071 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.523043    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"]
Nov 04 01:34:18.526269 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.526244    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"]
Nov 04 01:34:18.529766 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.529699    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m66qn"]
Nov 04 01:34:18.531461 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.531443    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"]
Nov 04 01:34:18.533140 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.533120    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:34:18.533552 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.533532    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.533552 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.533543    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.533725 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.533539    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.533939 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.533906    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534034 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.533965    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.534034 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534016    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534144 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534123    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534144 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534139    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.534250 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534222    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534367 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534349    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 01:34:18.534431 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534410    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:34:18.534431 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534414    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 01:34:18.534603 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534560    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534773 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534757    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 01:34:18.534841 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534797    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534841 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534822    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.534964 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534800    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.534964 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534763    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 01:34:18.534964 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534865    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 01:34:18.534964 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534946    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.535148 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.534990    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.535148 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.535059    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.535148 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.535090    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.536487 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.536419    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:34:18.536487 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.536455    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.536655 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.536596    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 01:34:18.543101 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.543082    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:34:18.586935 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.586910    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 01:34:18.589984 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.589966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:18.590089 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.589996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.590089 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.590153 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-stats-auth\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.590153 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.590228 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pzgb\" (UniqueName: \"kubernetes.io/projected/07ebf665-177c-48e7-969c-28057a4ff1a4-kube-api-access-4pzgb\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.590228 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9377de09-4111-4499-a1d7-62e407f76422-serving-cert\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.590228 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590216    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/081f052e-3e74-4fd5-8d12-d91aa017a999-serving-cert\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.590360 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.590412 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.590412 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp9mj\" (UniqueName: \"kubernetes.io/projected/c7411ec4-7d75-44ad-ae7c-fae47eee36f4-kube-api-access-qp9mj\") pod \"volume-data-source-validator-58c9f5f46d-4w7fx\" (UID: \"c7411ec4-7d75-44ad-ae7c-fae47eee36f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:18.590505 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-default-certificate\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.590505 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.590505 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.590505 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-nginx-conf\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8ec90134-2d9f-418f-98e5-877305e1a0b2-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590557    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.590594    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-config-volume\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.590612    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:18.590718 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74628\" (UniqueName: \"kubernetes.io/projected/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-kube-api-access-74628\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.591065 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.591065 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.590760    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.090740467 +0000 UTC m=+31.130668582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:18.591065 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47p5l\" (UniqueName: \"kubernetes.io/projected/928e6807-f385-42a2-89d5-e9ed3b1089cf-kube-api-access-47p5l\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.591065 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.591065 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.590965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.591065 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591063    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-trusted-ca\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-snapshots\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp9v2\" (UniqueName: \"kubernetes.io/projected/9377de09-4111-4499-a1d7-62e407f76422-kube-api-access-hp9v2\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mc9h\" (UniqueName: \"kubernetes.io/projected/081f052e-3e74-4fd5-8d12-d91aa017a999-kube-api-access-6mc9h\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df3c83fd-42b4-423d-a2a2-9da642a21118-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-tmp-dir\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591295    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9km5\" (UniqueName: \"kubernetes.io/projected/8ec90134-2d9f-418f-98e5-877305e1a0b2-kube-api-access-q9km5\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.591364 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.591756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr76c\" (UniqueName: \"kubernetes.io/projected/df3c83fd-42b4-423d-a2a2-9da642a21118-kube-api-access-nr76c\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.591756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591468    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.591756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591490    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.591756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.591756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.591756 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/081f052e-3e74-4fd5-8d12-d91aa017a999-config\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.591935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67jjd\" (UniqueName: \"kubernetes.io/projected/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db-kube-api-access-67jjd\") pod \"kas-connection-checker-685895c7c9-sx466\" (UID: \"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db\") " pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.591959    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.09194313 +0000 UTC m=+31.131871245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvxs\" (UniqueName: \"kubernetes.io/projected/81cc5d87-5a2f-43e5-ba84-04993c15476e-kube-api-access-2qvxs\") pod \"kas-connection-checker-685895c7c9-j4l4l\" (UID: \"81cc5d87-5a2f-43e5-ba84-04993c15476e\") " pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w947m\" (UniqueName: \"kubernetes.io/projected/9b16b031-58ec-4d17-82b1-61a7c952c887-kube-api-access-w947m\") pod \"network-check-source-6f78d59f99-8hlvs\" (UID: \"9b16b031-58ec-4d17-82b1-61a7c952c887\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:18.592141 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.592524 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df3c83fd-42b4-423d-a2a2-9da642a21118-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.592524 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87w7x\" (UniqueName: \"kubernetes.io/projected/b546556a-5219-406b-88d0-a3b490c7b259-kube-api-access-87w7x\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.592524 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d5f\" (UniqueName: \"kubernetes.io/projected/6a1464fb-a344-4bd9-8406-6fe464ff4fa0-kube-api-access-l4d5f\") pod \"kas-connection-checker-685895c7c9-f66h9\" (UID: \"6a1464fb-a344-4bd9-8406-6fe464ff4fa0\") " pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:18.592524 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592391    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-snapshots\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.592710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-config\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.592710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592641    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-service-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.592854 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.592833    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/081f052e-3e74-4fd5-8d12-d91aa017a999-config\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.593163 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.593140    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-service-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.593224 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.593173    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df3c83fd-42b4-423d-a2a2-9da642a21118-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.593368 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.593350    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:18.593427 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.593370    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:18.593427 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.593372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.593610    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.093557197 +0000 UTC m=+31.133485314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594040    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b546556a-5219-406b-88d0-a3b490c7b259-serving-cert\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5z6n\" (UniqueName: \"kubernetes.io/projected/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-kube-api-access-j5z6n\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594399    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-cert\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-tmp\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594850    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-tmp\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.594974 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.594934    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8ec90134-2d9f-418f-98e5-877305e1a0b2-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.595442 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.595128    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.595442 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.595181    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.595442 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.595288    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.596363 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.596342    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.627642 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.627617    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:34:18.627642 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.627627    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 01:34:18.627642 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.627632    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.628323    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.628607    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.629540    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.629993    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"]
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.630888    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.630929    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631047    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631154    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ldpmc\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631220    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631336    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-4rqhb\""
Nov 04 01:34:18.631550 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631419    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xkhwh\""
Nov 04 01:34:18.632113 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631623    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gnb5c\""
Nov 04 01:34:18.632113 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.631627    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:18.632113 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.631705    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.131678286 +0000 UTC m=+31.171606380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:18.632113 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631927    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vqsf7\""
Nov 04 01:34:18.632113 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.631993    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 01:34:18.632113 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632054    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 01:34:18.632453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632124    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 01:34:18.632453 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.632141    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:18.632453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632185    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 01:34:18.632453 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.632367    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.132234144 +0000 UTC m=+31.172164437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:18.632453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632417    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:34:18.632694 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632453    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 01:34:18.632694 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632532    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mh5l6\""
Nov 04 01:34:18.632694 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.632594    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:34:18.633036 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.633017    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.634334 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.633914    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"]
Nov 04 01:34:18.634493 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.634469    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.635005 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.634955    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.635143 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.635040    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9377de09-4111-4499-a1d7-62e407f76422-serving-cert\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.636748 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.635263    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-stats-auth\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.636748 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.636020    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-default-certificate\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.636748 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.636087    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jn9bb"]
Nov 04 01:34:18.636748 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.636534    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/081f052e-3e74-4fd5-8d12-d91aa017a999-serving-cert\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.636900 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.636847    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54848578cf-gq6dc"]
Nov 04 01:34:18.636931 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.636900    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df3c83fd-42b4-423d-a2a2-9da642a21118-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.642454 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.642438    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:18.642515 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.642486    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.142472355 +0000 UTC m=+31.182400448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:18.652907 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.652878    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pzgb\" (UniqueName: \"kubernetes.io/projected/07ebf665-177c-48e7-969c-28057a4ff1a4-kube-api-access-4pzgb\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:18.653006 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.652934    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp9mj\" (UniqueName: \"kubernetes.io/projected/c7411ec4-7d75-44ad-ae7c-fae47eee36f4-kube-api-access-qp9mj\") pod \"volume-data-source-validator-58c9f5f46d-4w7fx\" (UID: \"c7411ec4-7d75-44ad-ae7c-fae47eee36f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:18.653006 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.652979    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.653945 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.653927    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47p5l\" (UniqueName: \"kubernetes.io/projected/928e6807-f385-42a2-89d5-e9ed3b1089cf-kube-api-access-47p5l\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:18.653945 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.653941    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67jjd\" (UniqueName: \"kubernetes.io/projected/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db-kube-api-access-67jjd\") pod \"kas-connection-checker-685895c7c9-sx466\" (UID: \"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db\") " pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:18.659464 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.659443    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:18.659623 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.659605    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9km5\" (UniqueName: \"kubernetes.io/projected/8ec90134-2d9f-418f-98e5-877305e1a0b2-kube-api-access-q9km5\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:18.659814 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.659795    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp9v2\" (UniqueName: \"kubernetes.io/projected/9377de09-4111-4499-a1d7-62e407f76422-kube-api-access-hp9v2\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.661580 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.661553    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.661906 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.661881    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvxs\" (UniqueName: \"kubernetes.io/projected/81cc5d87-5a2f-43e5-ba84-04993c15476e-kube-api-access-2qvxs\") pod \"kas-connection-checker-685895c7c9-j4l4l\" (UID: \"81cc5d87-5a2f-43e5-ba84-04993c15476e\") " pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:18.662112 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.662096    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4d5f\" (UniqueName: \"kubernetes.io/projected/6a1464fb-a344-4bd9-8406-6fe464ff4fa0-kube-api-access-l4d5f\") pod \"kas-connection-checker-685895c7c9-f66h9\" (UID: \"6a1464fb-a344-4bd9-8406-6fe464ff4fa0\") " pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:18.664420 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.664401    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:18.664749 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.664731    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mc9h\" (UniqueName: \"kubernetes.io/projected/081f052e-3e74-4fd5-8d12-d91aa017a999-kube-api-access-6mc9h\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.664925 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.664910    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr76c\" (UniqueName: \"kubernetes.io/projected/df3c83fd-42b4-423d-a2a2-9da642a21118-kube-api-access-nr76c\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.676874 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.676829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:18.693777 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.693759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:18.694999 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.694984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-config\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.695085 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b546556a-5219-406b-88d0-a3b490c7b259-serving-cert\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.695085 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5z6n\" (UniqueName: \"kubernetes.io/projected/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-kube-api-access-j5z6n\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.695167 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-cert\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.695167 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695111    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:18.695245 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-nginx-conf\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:18.695304 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-config-volume\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.695347 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.695329    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:18.695395 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695332    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74628\" (UniqueName: \"kubernetes.io/projected/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-kube-api-access-74628\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.695439 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.695393    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.195368368 +0000 UTC m=+31.235296480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:18.695439 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-trusted-ca\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.695524 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-tmp-dir\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.695524 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.695514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696209    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-nginx-conf\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.696341    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696361    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-config-volume\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w947m\" (UniqueName: \"kubernetes.io/projected/9b16b031-58ec-4d17-82b1-61a7c952c887-kube-api-access-w947m\") pod \"network-check-source-6f78d59f99-8hlvs\" (UID: \"9b16b031-58ec-4d17-82b1-61a7c952c887\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:18.696395    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls podName:d3b79cfe-4f1c-4786-9f97-e87ce36cbba8 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.196379164 +0000 UTC m=+31.236307276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls") pod "dns-default-jn9bb" (UID: "d3b79cfe-4f1c-4786-9f97-e87ce36cbba8") : secret "dns-default-metrics-tls" not found
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696343    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-tmp-dir\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696434    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-config\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.696453 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696434    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-trusted-ca\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.697012 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.696460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87w7x\" (UniqueName: \"kubernetes.io/projected/b546556a-5219-406b-88d0-a3b490c7b259-kube-api-access-87w7x\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.697870 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.697850    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b546556a-5219-406b-88d0-a3b490c7b259-serving-cert\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.698221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.698194    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-cert\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.721783 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.721757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:18.733891 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.733862    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:18.740323 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.740258    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:18.760186 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.758427    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5z6n\" (UniqueName: \"kubernetes.io/projected/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-kube-api-access-j5z6n\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:18.760186 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.758910    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87w7x\" (UniqueName: \"kubernetes.io/projected/b546556a-5219-406b-88d0-a3b490c7b259-kube-api-access-87w7x\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.760186 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.759912    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w947m\" (UniqueName: \"kubernetes.io/projected/9b16b031-58ec-4d17-82b1-61a7c952c887-kube-api-access-w947m\") pod \"network-check-source-6f78d59f99-8hlvs\" (UID: \"9b16b031-58ec-4d17-82b1-61a7c952c887\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:18.760186 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.760050    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74628\" (UniqueName: \"kubernetes.io/projected/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-kube-api-access-74628\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.767404 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.767378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:18.773846 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.773808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:18.788309 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.788276    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:18.834400 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.834037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:18.843852 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.843654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:18.852597 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.852384    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:18.984443 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:18.984415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:19.077561 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.077530    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-f66h9"]
Nov 04 01:34:19.080213 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.080189    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1464fb_a344_4bd9_8406_6fe464ff4fa0.slice/crio-1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c WatchSource:0}: Error finding container 1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c: Status 404 returned error can't find the container with id 1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c
Nov 04 01:34:19.100347 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.100154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:19.100437 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.100409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:19.100437 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100300    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:19.100543 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100442    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:19.100543 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.100490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:19.100543 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100505    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.10048442 +0000 UTC m=+32.140412517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:19.100713 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100596    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:19.100713 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100610    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:19.100713 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100612    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.100594182 +0000 UTC m=+32.140522276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:19.100713 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.100643    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.100632318 +0000 UTC m=+32.140560450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:19.103418 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.103395    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-j4l4l"]
Nov 04 01:34:19.106090 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.106070    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81cc5d87_5a2f_43e5_ba84_04993c15476e.slice/crio-df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190 WatchSource:0}: Error finding container df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190: Status 404 returned error can't find the container with id df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190
Nov 04 01:34:19.131506 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.131439    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-sx466"]
Nov 04 01:34:19.136338 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.136308    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc699c1_3d13_46f0_8f81_fcb9f7c7a1db.slice/crio-2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9 WatchSource:0}: Error finding container 2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9: Status 404 returned error can't find the container with id 2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9
Nov 04 01:34:19.142733 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.142613    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"]
Nov 04 01:34:19.171458 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.171428    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"]
Nov 04 01:34:19.176370 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.176343    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3c83fd_42b4_423d_a2a2_9da642a21118.slice/crio-c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e WatchSource:0}: Error finding container c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e: Status 404 returned error can't find the container with id c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e
Nov 04 01:34:19.193778 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.193049    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8zdbz"]
Nov 04 01:34:19.196965 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.196888    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"]
Nov 04 01:34:19.203485 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.203446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:19.203546 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.203497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:19.203546 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.203536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:19.203682 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203661    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:19.203682 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203666    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.203688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203661    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203712    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.203692243 +0000 UTC m=+32.243620367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203738    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.203728658 +0000 UTC m=+32.243656754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203750    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203755    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.203746521 +0000 UTC m=+32.243674617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203779    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls podName:d3b79cfe-4f1c-4786-9f97-e87ce36cbba8 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.203770617 +0000 UTC m=+32.243698714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls") pod "dns-default-jn9bb" (UID: "d3b79cfe-4f1c-4786-9f97-e87ce36cbba8") : secret "dns-default-metrics-tls" not found
Nov 04 01:34:19.203868 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.203839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:19.204304 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203890    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:19.204304 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:19.203921    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:20.203913471 +0000 UTC m=+32.243841563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:19.242533 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.242473    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"]
Nov 04 01:34:19.245492 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.245464    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb546556a_5219_406b_88d0_a3b490c7b259.slice/crio-e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc WatchSource:0}: Error finding container e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc: Status 404 returned error can't find the container with id e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc
Nov 04 01:34:19.248097 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.248027    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"]
Nov 04 01:34:19.249905 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.249880    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b16b031_58ec_4d17_82b1_61a7c952c887.slice/crio-b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f WatchSource:0}: Error finding container b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f: Status 404 returned error can't find the container with id b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f
Nov 04 01:34:19.262221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.262203    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m66qn"]
Nov 04 01:34:19.263697 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:19.263677    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f41073_2c64_44f1_9153_ca4e0e2c0b50.slice/crio-6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd WatchSource:0}: Error finding container 6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd: Status 404 returned error can't find the container with id 6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd
Nov 04 01:34:19.640834 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.640802    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m66qn" event={"ID":"d4f41073-2c64-44f1-9153-ca4e0e2c0b50","Type":"ContainerStarted","Data":"6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd"}
Nov 04 01:34:19.642055 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.642006    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs" event={"ID":"9b16b031-58ec-4d17-82b1-61a7c952c887","Type":"ContainerStarted","Data":"b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f"}
Nov 04 01:34:19.642991 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.642972    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" event={"ID":"b546556a-5219-406b-88d0-a3b490c7b259","Type":"ContainerStarted","Data":"e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc"}
Nov 04 01:34:19.645990 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.645969    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx" event={"ID":"c7411ec4-7d75-44ad-ae7c-fae47eee36f4","Type":"ContainerStarted","Data":"dff8563496a83adbf36169bd514839a31531f5745541706700246c18c7e87a4e"}
Nov 04 01:34:19.647537 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.647518    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c" event={"ID":"081f052e-3e74-4fd5-8d12-d91aa017a999","Type":"ContainerStarted","Data":"528629f5a4a1ed7cf0abb04404fb4ba7868511c0a1088ef695afeeaa28a2b164"}
Nov 04 01:34:19.649150 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.649126    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-f66h9" event={"ID":"6a1464fb-a344-4bd9-8406-6fe464ff4fa0","Type":"ContainerStarted","Data":"556cd622aab5017888f553aec2164208d7e2e5c1151e45a501f29a86b64e4fd3"}
Nov 04 01:34:19.649241 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.649155    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-f66h9" event={"ID":"6a1464fb-a344-4bd9-8406-6fe464ff4fa0","Type":"ContainerStarted","Data":"1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c"}
Nov 04 01:34:19.650483 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.650464    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-sx466" event={"ID":"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db","Type":"ContainerStarted","Data":"7b446012ba83b4c2dc73cc1fdff07d1bd8461e16a4f5ee335cf3fdfbe1ca37dd"}
Nov 04 01:34:19.650553 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.650497    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-sx466" event={"ID":"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db","Type":"ContainerStarted","Data":"2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9"}
Nov 04 01:34:19.651699 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.651681    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-j4l4l" event={"ID":"81cc5d87-5a2f-43e5-ba84-04993c15476e","Type":"ContainerStarted","Data":"f18f0ecab43c5f60f9b6e1ad4fe59a5756713a019c3ab9ac21f37f1a8b1b3903"}
Nov 04 01:34:19.651777 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.651716    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-j4l4l" event={"ID":"81cc5d87-5a2f-43e5-ba84-04993c15476e","Type":"ContainerStarted","Data":"df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190"}
Nov 04 01:34:19.652754 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.652736    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8zdbz" event={"ID":"9377de09-4111-4499-a1d7-62e407f76422","Type":"ContainerStarted","Data":"14760a4a284308dd6c057cec769f6709f61d2bb262e9d6f6cb62a6f838ee9a93"}
Nov 04 01:34:19.653744 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:19.653727    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4" event={"ID":"df3c83fd-42b4-423d-a2a2-9da642a21118","Type":"ContainerStarted","Data":"c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e"}
Nov 04 01:34:20.112163 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.112137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:20.112319 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.112185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:20.112319 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.112246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:20.112319 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112281    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:20.112319 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112297    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:20.112432 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112341    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.112328028 +0000 UTC m=+34.152256120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:20.112432 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112366    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.112352884 +0000 UTC m=+34.152280998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:20.112432 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112365    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:20.112432 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112385    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:20.112432 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.112408    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.112399962 +0000 UTC m=+34.152328055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:20.213367 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.213336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:20.213527 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.213399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:20.213527 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.213437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:20.213527 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.213482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:20.213834 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.213560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:20.213834 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213720    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:20.213834 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213768    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.213756479 +0000 UTC m=+34.253684584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:20.213834 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213810    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:20.213834 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213830    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.213824033 +0000 UTC m=+34.253752138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:20.214137 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213860    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:20.214137 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213878    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.213872416 +0000 UTC m=+34.253800521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:20.214137 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213907    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:20.214137 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213925    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.213919466 +0000 UTC m=+34.253847571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:20.214137 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213969    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:34:20.214137 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:20.213989    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls podName:d3b79cfe-4f1c-4786-9f97-e87ce36cbba8 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:22.213983486 +0000 UTC m=+34.253911591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls") pod "dns-default-jn9bb" (UID: "d3b79cfe-4f1c-4786-9f97-e87ce36cbba8") : secret "dns-default-metrics-tls" not found
Nov 04 01:34:20.629743 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.629692    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:20.757162 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.757112    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-sx466" podStartSLOduration=413.757097344 podStartE2EDuration="6m53.757097344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:20.638696241 +0000 UTC m=+32.678624355" watchObservedRunningTime="2026-11-04 01:34:20.757097344 +0000 UTC m=+32.797025456"
Nov 04 01:34:20.875234 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.875183    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-f66h9" podStartSLOduration=413.875167311 podStartE2EDuration="6m53.875167311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:20.757595939 +0000 UTC m=+32.797524058" watchObservedRunningTime="2026-11-04 01:34:20.875167311 +0000 UTC m=+32.915095456"
Nov 04 01:34:20.879849 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:20.879781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:22.138786 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.138740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.138861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.138910    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139081    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.139067099 +0000 UTC m=+38.178995191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139164    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139174    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139201    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.139191143 +0000 UTC m=+38.179119236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139248    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139256    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:22.139274 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.139274    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.139268566 +0000 UTC m=+38.179196658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:22.239658 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.239582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:22.239814 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.239677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:22.239814 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.239717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:22.239814 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.239742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:22.239814 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.239770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:22.240021 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.239883    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:34:22.240021 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.239927    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls podName:d3b79cfe-4f1c-4786-9f97-e87ce36cbba8 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.239914257 +0000 UTC m=+38.279842362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls") pod "dns-default-jn9bb" (UID: "d3b79cfe-4f1c-4786-9f97-e87ce36cbba8") : secret "dns-default-metrics-tls" not found
Nov 04 01:34:22.240021 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.239967    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:22.240021 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.239987    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.239980873 +0000 UTC m=+38.279908965 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:22.240021 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.240018    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:22.240266 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.240034    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.240029495 +0000 UTC m=+38.279957600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:22.240266 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.240063    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:22.240266 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.240083    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.240076224 +0000 UTC m=+38.280004316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:22.240266 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.240124    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:22.240266 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.240152    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.24014334 +0000 UTC m=+38.280071445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:22.341697 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.341218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:22.341697 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.341319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:22.341697 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.341377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:22.341697 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.341552    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:34:22.341697 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:22.341636    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:54.341615748 +0000 UTC m=+66.381543857 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : secret "metrics-daemon-secret" not found
Nov 04 01:34:22.347735 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.347678    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:22.353858 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.353800    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:22.403422 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.403130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:22.408040 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.407874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:22.676724 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.675956    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx" event={"ID":"c7411ec4-7d75-44ad-ae7c-fae47eee36f4","Type":"ContainerStarted","Data":"fde546a7a8fe257dfe3514b28dfbfae344c4171a92eda8e9c9162124462e34b0"}
Nov 04 01:34:22.795758 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.795652    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-j4l4l" podStartSLOduration=415.795637016 podStartE2EDuration="6m55.795637016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:20.875511548 +0000 UTC m=+32.915439661" watchObservedRunningTime="2026-11-04 01:34:22.795637016 +0000 UTC m=+34.835565127"
Nov 04 01:34:22.814045 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.813986    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx" podStartSLOduration=383.0629751 podStartE2EDuration="6m25.813966182s" totalImagesPullingTime="2.750991082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:22.795886545 +0000 UTC m=+34.835814650" watchObservedRunningTime="2026-11-04 01:34:22.813966182 +0000 UTC m=+34.853894297"
Nov 04 01:34:22.815149 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.815125    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-68r2l"]
Nov 04 01:34:22.851987 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:22.851959    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5bea9b9_e510_4b8b_bc95_5ad38581b40c.slice/crio-b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3 WatchSource:0}: Error finding container b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3: Status 404 returned error can't find the container with id b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3
Nov 04 01:34:22.862388 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:22.862333    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p7jkf"]
Nov 04 01:34:22.869017 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:22.868993    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b11319_689c_4ba5_83ec_afe254b80174.slice/crio-a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3 WatchSource:0}: Error finding container a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3: Status 404 returned error can't find the container with id a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3
Nov 04 01:34:23.007530 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.007488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:23.264761 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.264670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:23.682939 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.682900    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8zdbz" event={"ID":"9377de09-4111-4499-a1d7-62e407f76422","Type":"ContainerStarted","Data":"f0615d2882a76e671de900b632848e274e81bd120813919af48abaed3d11d57d"}
Nov 04 01:34:23.685418 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.685395    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4" event={"ID":"df3c83fd-42b4-423d-a2a2-9da642a21118","Type":"ContainerStarted","Data":"bf060721ec495af2cd42411cfae8ee90f277b5b04111d22e79a020d959eb4a8e"}
Nov 04 01:34:23.687256 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.687202    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p7jkf" event={"ID":"76b11319-689c-4ba5-83ec-afe254b80174","Type":"ContainerStarted","Data":"a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3"}
Nov 04 01:34:23.689221 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.689168    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-68r2l" event={"ID":"a5bea9b9-e510-4b8b-bc95-5ad38581b40c","Type":"ContainerStarted","Data":"b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3"}
Nov 04 01:34:23.691584 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.691544    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/0.log"
Nov 04 01:34:23.691673 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.691598    2576 generic.go:356] "Generic (PLEG): container finished" podID="b546556a-5219-406b-88d0-a3b490c7b259" containerID="8c8b09b2b728dc56fb5ca85ba5d89d00a88c65c100e9f2140668a089d9c6fab2" exitCode=255
Nov 04 01:34:23.692233 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.692211    2576 scope.go:122] "RemoveContainer" containerID="8c8b09b2b728dc56fb5ca85ba5d89d00a88c65c100e9f2140668a089d9c6fab2"
Nov 04 01:34:23.692474 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.692454    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" event={"ID":"b546556a-5219-406b-88d0-a3b490c7b259","Type":"ContainerDied","Data":"8c8b09b2b728dc56fb5ca85ba5d89d00a88c65c100e9f2140668a089d9c6fab2"}
Nov 04 01:34:23.806796 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.806682    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-8zdbz" podStartSLOduration=380.575852706 podStartE2EDuration="6m24.806665879s" totalImagesPullingTime="4.230813173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:23.806049733 +0000 UTC m=+35.845977847" watchObservedRunningTime="2026-11-04 01:34:23.806665879 +0000 UTC m=+35.846593993"
Nov 04 01:34:23.926854 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:23.926797    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4" podStartSLOduration=382.595433473 podStartE2EDuration="6m26.926776964s" totalImagesPullingTime="4.331343491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:23.924959083 +0000 UTC m=+35.964887197" watchObservedRunningTime="2026-11-04 01:34:23.926776964 +0000 UTC m=+35.966705079"
Nov 04 01:34:24.697665 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.697557    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/1.log"
Nov 04 01:34:24.699646 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.698421    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/0.log"
Nov 04 01:34:24.699646 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.698459    2576 generic.go:356] "Generic (PLEG): container finished" podID="b546556a-5219-406b-88d0-a3b490c7b259" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0" exitCode=255
Nov 04 01:34:24.699646 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.698531    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" event={"ID":"b546556a-5219-406b-88d0-a3b490c7b259","Type":"ContainerDied","Data":"99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0"}
Nov 04 01:34:24.699646 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.698605    2576 scope.go:122] "RemoveContainer" containerID="8c8b09b2b728dc56fb5ca85ba5d89d00a88c65c100e9f2140668a089d9c6fab2"
Nov 04 01:34:24.699646 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.698968    2576 scope.go:122] "RemoveContainer" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0"
Nov 04 01:34:24.699646 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:24.699251    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wsc5z_openshift-console-operator(b546556a-5219-406b-88d0-a3b490c7b259)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259"
Nov 04 01:34:24.704056 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.702954    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c" event={"ID":"081f052e-3e74-4fd5-8d12-d91aa017a999","Type":"ContainerStarted","Data":"2e25156a42e00345b09abf30c20c5576ee5674cbf3fe3badba37b5744a5f8805"}
Nov 04 01:34:24.872743 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.872705    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6xjwj"]
Nov 04 01:34:24.877171 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.876229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:24.936579 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.936529    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xjwj"]
Nov 04 01:34:24.936917 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.936888    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j6ln8\""
Nov 04 01:34:24.937047 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.936968    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:34:24.937047 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.937029    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:34:24.942415 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.941318    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c" podStartSLOduration=379.228796863 podStartE2EDuration="6m23.941301992s" totalImagesPullingTime="4.712505129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:24.940457246 +0000 UTC m=+36.980385382" watchObservedRunningTime="2026-11-04 01:34:24.941301992 +0000 UTC m=+36.981230108"
Nov 04 01:34:24.969473 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.969121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/909efb32-95fc-4b85-bf58-949637c49c16-crio-socket\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:24.969473 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.969162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/909efb32-95fc-4b85-bf58-949637c49c16-data-volume\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:24.969473 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.969199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:24.969473 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.969226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:24.969473 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:24.969278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcbz6\" (UniqueName: \"kubernetes.io/projected/909efb32-95fc-4b85-bf58-949637c49c16-kube-api-access-jcbz6\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.070665 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.070600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.070851 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.070717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbz6\" (UniqueName: \"kubernetes.io/projected/909efb32-95fc-4b85-bf58-949637c49c16-kube-api-access-jcbz6\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.070851 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.070811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/909efb32-95fc-4b85-bf58-949637c49c16-crio-socket\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.070969 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.070855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/909efb32-95fc-4b85-bf58-949637c49c16-data-volume\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.070969 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.070899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.071080 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.071036    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:25.071129 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.071097    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:25.571079444 +0000 UTC m=+37.611007554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:25.071363 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.071344    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:34:25.071462 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.071407    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:25.571393762 +0000 UTC m=+37.611321876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : configmap "kube-rbac-proxy" not found
Nov 04 01:34:25.071708 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.071686    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/909efb32-95fc-4b85-bf58-949637c49c16-crio-socket\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.072042 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.072015    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/909efb32-95fc-4b85-bf58-949637c49c16-data-volume\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.138586 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.131956    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-ttvxq-rrlpl"]
Nov 04 01:34:25.138586 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.132755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:25.140286 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.140262    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbz6\" (UniqueName: \"kubernetes.io/projected/909efb32-95fc-4b85-bf58-949637c49c16-kube-api-access-jcbz6\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.142980 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.142963    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.187930 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.187876    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"]
Nov 04 01:34:25.190786 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.190067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:25.200278 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.200209    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-ttvxq-rrlpl"]
Nov 04 01:34:25.210262 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.210226    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r7ngl\""
Nov 04 01:34:25.225407 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.225347    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"]
Nov 04 01:34:25.227519 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.227500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:25.248130 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.248087    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:25.249061 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.249008    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 01:34:25.252138 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.252106    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"]
Nov 04 01:34:25.273124 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.272685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.273124 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.272863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.273124 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.272927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.273124 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.272994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.274956 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.274811    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"]
Nov 04 01:34:25.308049 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.308004    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9rdvr\""
Nov 04 01:34:25.374522 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.374494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.374710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.374544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.374710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.374598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tld9\" (UniqueName: \"kubernetes.io/projected/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea-kube-api-access-4tld9\") pod \"migrator-66df5d496b-42wch\" (UID: \"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:25.374710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.374641    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.374710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.374691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.374710 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.374708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-692kd\" (UniqueName: \"kubernetes.io/projected/d5017035-43de-4aec-bcb7-cbdee8be6bc3-kube-api-access-692kd\") pod \"migrator-66df5d496b-97prq\" (UID: \"d5017035-43de-4aec-bcb7-cbdee8be6bc3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:25.375120 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.375101    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.376051 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.376006    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.376051 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.376043    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.399320 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.399286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:25.437987 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.437942    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.458618 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.458588    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:25.480089 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.479118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tld9\" (UniqueName: \"kubernetes.io/projected/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea-kube-api-access-4tld9\") pod \"migrator-66df5d496b-42wch\" (UID: \"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:25.480089 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.479214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-692kd\" (UniqueName: \"kubernetes.io/projected/d5017035-43de-4aec-bcb7-cbdee8be6bc3-kube-api-access-692kd\") pod \"migrator-66df5d496b-97prq\" (UID: \"d5017035-43de-4aec-bcb7-cbdee8be6bc3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:25.548836 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.548806    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-692kd\" (UniqueName: \"kubernetes.io/projected/d5017035-43de-4aec-bcb7-cbdee8be6bc3-kube-api-access-692kd\") pod \"migrator-66df5d496b-97prq\" (UID: \"d5017035-43de-4aec-bcb7-cbdee8be6bc3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:25.549024 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.548851    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tld9\" (UniqueName: \"kubernetes.io/projected/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea-kube-api-access-4tld9\") pod \"migrator-66df5d496b-42wch\" (UID: \"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:25.580046 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.580006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.580194 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.580062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:25.580194 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.580167    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:25.580307 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.580212    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:34:25.580307 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.580249    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.580227312 +0000 UTC m=+38.620155422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:25.580307 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.580270    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:26.580257572 +0000 UTC m=+38.620185685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : configmap "kube-rbac-proxy" not found
Nov 04 01:34:25.647328 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.647300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:25.707962 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.707239    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m66qn" event={"ID":"d4f41073-2c64-44f1-9153-ca4e0e2c0b50","Type":"ContainerStarted","Data":"758dd54c1ba5ae91312a13974491f6862a2d5c8effe8f29c43bae81b31cdba1a"}
Nov 04 01:34:25.710384 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.710355    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs" event={"ID":"9b16b031-58ec-4d17-82b1-61a7c952c887","Type":"ContainerStarted","Data":"f0a8839c69195b102733d77efac3910c8b8b91a03602af00be9ec3ffe262a6fb"}
Nov 04 01:34:25.713146 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.713126    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/1.log"
Nov 04 01:34:25.713828 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.713813    2576 scope.go:122] "RemoveContainer" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0"
Nov 04 01:34:25.713997 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:25.713977    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wsc5z_openshift-console-operator(b546556a-5219-406b-88d0-a3b490c7b259)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259"
Nov 04 01:34:25.804616 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.804366    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:25.826335 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.826276    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m66qn" podStartSLOduration=2.015952137 podStartE2EDuration="7.826256487s" totalImagesPullingTime="5.81030435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:25.824589299 +0000 UTC m=+37.864517406" watchObservedRunningTime="2026-11-04 01:34:25.826256487 +0000 UTC m=+37.866184602"
Nov 04 01:34:25.838219 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.838024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:25.845173 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.845148    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-ttvxq-rrlpl"]
Nov 04 01:34:25.855524 ip-10-0-140-18 kubenswrapper[2576]: W1104 01:34:25.855496    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4164bd42_81bf_478d_8409_001aed0783dc.slice/crio-de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a WatchSource:0}: Error finding container de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a: Status 404 returned error can't find the container with id de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a
Nov 04 01:34:25.905924 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:25.905900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:26.162935 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.162902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:26.186195 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.186166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:26.186404 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.186232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:26.186404 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186333    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:26.186404 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.186350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:26.186404 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186357    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:26.186640 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186425    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.186401989 +0000 UTC m=+46.226330105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:26.186640 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186451    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.186441228 +0000 UTC m=+46.226369326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:26.186640 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186473    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:26.186640 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186485    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:26.186640 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.186535    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.186519952 +0000 UTC m=+46.226448062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:26.192465 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.192375    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs" podStartSLOduration=378.797503469 podStartE2EDuration="6m25.192358386s" totalImagesPullingTime="6.394854917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:26.073832356 +0000 UTC m=+38.113760470" watchObservedRunningTime="2026-11-04 01:34:26.192358386 +0000 UTC m=+38.232286505"
Nov 04 01:34:26.193307 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.193290    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"]
Nov 04 01:34:26.228640 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.228606    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"]
Nov 04 01:34:26.288829 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.288801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:26.288952 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.288866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:26.288952 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.288901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:26.288952 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.288937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:26.289106 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.288992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:26.289155 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289132    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:26.289199 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289184    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.289167796 +0000 UTC m=+46.329095890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289246    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289276    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.289269237 +0000 UTC m=+46.329197329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289316    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289332    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289342    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls podName:d3b79cfe-4f1c-4786-9f97-e87ce36cbba8 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.289335222 +0000 UTC m=+46.329263315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls") pod "dns-default-jn9bb" (UID: "d3b79cfe-4f1c-4786-9f97-e87ce36cbba8") : secret "dns-default-metrics-tls" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289377    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289381    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.289364434 +0000 UTC m=+46.329292558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:26.289406 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.289396    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.289390778 +0000 UTC m=+46.329318870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:26.592946 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.592912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:26.593125 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.592971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:26.593125 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.593092    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:26.593125 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.593105    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:34:26.593271 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.593167    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:28.59314961 +0000 UTC m=+40.633077702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : configmap "kube-rbac-proxy" not found
Nov 04 01:34:26.593271 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:26.593187    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:28.59317752 +0000 UTC m=+40.633105617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:26.600923 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.600894    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-tlb6z"]
Nov 04 01:34:26.603099 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.603079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.656484 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.656450    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 01:34:26.656732 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.656707    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rx6cn\""
Nov 04 01:34:26.662167 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.662144    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-tlb6z"]
Nov 04 01:34:26.664517 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.664498    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 01:34:26.664641 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.664601    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 01:34:26.664641 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.664625    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 01:34:26.718125 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.718060    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch" event={"ID":"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea","Type":"ContainerStarted","Data":"b38800075d1133e8454aa1676444a67e422371d4de0e8c321910cbc182fb4431"}
Nov 04 01:34:26.719464 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.719414    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq" event={"ID":"d5017035-43de-4aec-bcb7-cbdee8be6bc3","Type":"ContainerStarted","Data":"c24fe813338fab33377a99670a97eae5bca619cd8a19fd00efb058cb80cc38f7"}
Nov 04 01:34:26.721260 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.721229    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl" event={"ID":"4164bd42-81bf-478d-8409-001aed0783dc","Type":"ContainerStarted","Data":"7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991"}
Nov 04 01:34:26.721388 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.721273    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl" event={"ID":"4164bd42-81bf-478d-8409-001aed0783dc","Type":"ContainerStarted","Data":"de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a"}
Nov 04 01:34:26.722958 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.722929    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-68r2l" event={"ID":"a5bea9b9-e510-4b8b-bc95-5ad38581b40c","Type":"ContainerStarted","Data":"292c1089e412dfa181596fe3a23a44437e27ea8dd38bc0931c19f33cb95e7d61"}
Nov 04 01:34:26.723130 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.723115    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:26.795215 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.795183    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-cabundle\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.795425 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.795231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-key\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.795425 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.795264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw9md\" (UniqueName: \"kubernetes.io/projected/bd5494d4-4343-4f18-b981-b39fe6a54034-kube-api-access-mw9md\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.839314 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.839262    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl" podStartSLOduration=1.839247663 podStartE2EDuration="1.839247663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:26.838440325 +0000 UTC m=+38.878368439" watchObservedRunningTime="2026-11-04 01:34:26.839247663 +0000 UTC m=+38.879175777"
Nov 04 01:34:26.896906 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.896825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-cabundle\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.896906 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.896887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-key\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.897115 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.896918    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw9md\" (UniqueName: \"kubernetes.io/projected/bd5494d4-4343-4f18-b981-b39fe6a54034-kube-api-access-mw9md\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.897668 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.897642    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-cabundle\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.899711 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.899693    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-key\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:26.914185 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.914158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:26.955471 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.955416    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-68r2l" podStartSLOduration=34.608449366 podStartE2EDuration="37.955401838s" totalImagesPullingTime="3.346952472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:26.954842473 +0000 UTC m=+38.994770594" watchObservedRunningTime="2026-11-04 01:34:26.955401838 +0000 UTC m=+38.995329952"
Nov 04 01:34:26.962782 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:26.962755    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw9md\" (UniqueName: \"kubernetes.io/projected/bd5494d4-4343-4f18-b981-b39fe6a54034-kube-api-access-mw9md\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:27.166806 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.166704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:27.214664 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.214409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:27.606789 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.606758    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-tlb6z"]
Nov 04 01:34:27.729678 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.729635    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z" event={"ID":"bd5494d4-4343-4f18-b981-b39fe6a54034","Type":"ContainerStarted","Data":"0ce53d92f3bdf326f2c0053eb67a42864bfac5b9f3faa9d4ec00614df500bd46"}
Nov 04 01:34:27.730113 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.729697    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z" event={"ID":"bd5494d4-4343-4f18-b981-b39fe6a54034","Type":"ContainerStarted","Data":"f7dd48e88a5e0b780d6de985441712a8e0ef3f18bac3c9224c2b7194c900b5b0"}
Nov 04 01:34:27.851941 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.851887    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z" podStartSLOduration=1.8518717900000001 podStartE2EDuration="1.85187179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:27.850881091 +0000 UTC m=+39.890809206" watchObservedRunningTime="2026-11-04 01:34:27.85187179 +0000 UTC m=+39.891799903"
Nov 04 01:34:27.983973 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:27.983906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:28.614142 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.614100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:28.614328 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:28.614235    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:34:28.614328 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:28.614310    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.614289228 +0000 UTC m=+44.654217342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : configmap "kube-rbac-proxy" not found
Nov 04 01:34:28.614413 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.614381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:28.614505 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:28.614492    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:28.614554 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:28.614542    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.614529285 +0000 UTC m=+44.654457398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:28.736132 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.736086    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch" event={"ID":"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea","Type":"ContainerStarted","Data":"af2e8d18bab0cd16f6a0f987613ff3208cca5dd2e3817aa8b2860fcd2b827d41"}
Nov 04 01:34:28.737984 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.737952    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq" event={"ID":"d5017035-43de-4aec-bcb7-cbdee8be6bc3","Type":"ContainerStarted","Data":"96d04dbf4f95137dc7e36d2a518e3585212b4ce2a2f1e9f498550a3d0b535b8f"}
Nov 04 01:34:28.834757 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.834718    2576 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:28.834904 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.834782    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:28.835210 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.835193    2576 scope.go:122] "RemoveContainer" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0"
Nov 04 01:34:28.835471 ip-10-0-140-18 kubenswrapper[2576]: E1104 01:34:28.835449    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wsc5z_openshift-console-operator(b546556a-5219-406b-88d0-a3b490c7b259)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259"
Nov 04 01:34:28.854617 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.854509    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch" podStartSLOduration=2.079767852 podStartE2EDuration="3.854494663s" totalImagesPullingTime="1.774726811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:28.854431762 +0000 UTC m=+40.894359878" watchObservedRunningTime="2026-11-04 01:34:28.854494663 +0000 UTC m=+40.894422800"
Nov 04 01:34:28.972172 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:28.972083    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq" podStartSLOduration=2.151876459 podStartE2EDuration="3.972063471s" totalImagesPullingTime="1.820187012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:28.971333192 +0000 UTC m=+41.011261306" watchObservedRunningTime="2026-11-04 01:34:28.972063471 +0000 UTC m=+41.011991585"
Nov 04 01:34:29.102870 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:29.102846    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzccz_e13536d8-6703-46a7-9e8e-8136e1ff8979/dns-node-resolver/0.log"
Nov 04 01:34:29.165217 ip-10-0-140-18 kubenswrapper[2576]: I1104 01:34:29.165181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:29.224789 ip-10-0-140-18 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 01:34:29.235102 ip-10-0-140-18 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 01:34:29.235275 ip-10-0-140-18 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 01:34:29.235462 ip-10-0-140-18 systemd[1]: kubelet.service: Consumed 3.266s CPU time.
Nov 04 01:34:29.250341 ip-10-0-140-18 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:34:29.313232 ip-10-0-140-18 kubenswrapper[5039]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:34:29.313232 ip-10-0-140-18 kubenswrapper[5039]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:34:29.313232 ip-10-0-140-18 kubenswrapper[5039]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:34:29.313232 ip-10-0-140-18 kubenswrapper[5039]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:34:29.315323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315240    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.315323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315324    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315328    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315331    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315334    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315337    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315339    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315342    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315345    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315347    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315350    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315352    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315355    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315357    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315361    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315365    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315368    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315378    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315390    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315393    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.315392 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315396    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315399    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315402    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315405    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315407    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315411    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315414    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315416    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315419    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315421    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315424    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315426    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315428    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315431    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315433    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315435    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315439    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315442    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315445    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.315858 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315448    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315450    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315452    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315455    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315457    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315461    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315463    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315465    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315468    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315471    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315473    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315476    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315483    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315486    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315489    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315491    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315493    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315496    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315498    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315501    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.316304 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315503    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315505    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315508    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315510    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315512    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315515    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315517    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315519    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315522    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315524    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315527    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315529    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315531    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315534    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315536    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315539    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315541    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315544    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315547    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.316802 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315549    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315551    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315554    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315556    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315559    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315561    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315588    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315591    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315594    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315596    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315598    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315601    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315604    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315606    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315608    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315611    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315613    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315615    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315618    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315620    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.317323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315623    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315625    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315627    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315630    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315632    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315635    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315637    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.315640    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316402    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316408    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316412    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316415    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316417    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316420    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316422    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316425    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316428    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316431    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316433    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.318089 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316436    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316439    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316441    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316444    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316446    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316449    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316451    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316453    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316456    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316458    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316461    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316463    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316465    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316468    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316470    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316473    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316475    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316478    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316480    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316483    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.318559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316486    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316490    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316493    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316499    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316501    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316504    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316507    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316509    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316511    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316514    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316516    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316519    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316521    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316524    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316526    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316529    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316531    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316534    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316536    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.319088 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316538    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316541    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316543    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316547    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316550    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316553    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316556    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316559    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316561    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316582    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316585    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316588    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316590    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316593    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316595    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316599    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316603    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316605    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316608    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.319542 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316610    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316613    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316615    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316617    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316620    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316622    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316625    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316627    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316630    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316633    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316635    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316638    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316640    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316643    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316645    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316647    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316650    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316652    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316666    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316668    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.320007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316671    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316674    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316677    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316679    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316682    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316685    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316687    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316690    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316692    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316696    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316700    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316703    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316705    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316708    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316710    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316713    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.316715    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317544    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317557    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317562    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.320521 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317586    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317591    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317597    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317601    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317612    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317616    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317621    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317627    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317633    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317638    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317642    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317647    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317652    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317656    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317660    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317665    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317669    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317677    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317682    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.321009 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317686    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317690    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317697    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317701    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317705    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317711    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317717    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317721    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317725    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317730    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317738    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317742    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317746    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317750    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317785    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317790    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317794    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317798    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317802    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.321445 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317805    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317809    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317813    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317817    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317825    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317829    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317835    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317839    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317843    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317847    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317851    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317855    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317858    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317862    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317866    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317872    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317876    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317885    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317889    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317893    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.321931 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317897    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317902    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317906    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317910    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317914    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317918    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317922    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317929    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317961    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317971    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317975    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317979    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317983    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317987    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317991    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317995    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.317999    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318003    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318008    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318012    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.322395 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318017    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318026    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318030    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318033    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318037    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318041    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318045    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318051    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318059    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318063    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318066    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318069    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318092    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318095    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318111    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318114    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318211    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318215    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318218    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.322881 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318221    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318223    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318226    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318228    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318233    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318235    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318340    5039 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318347    5039 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318354    5039 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318359    5039 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318363    5039 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318368    5039 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318372    5039 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318376    5039 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318380    5039 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318382    5039 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318386    5039 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318389    5039 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318392    5039 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318395    5039 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318399    5039 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318402    5039 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318404    5039 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:34:29.323303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318407    5039 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318411    5039 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318414    5039 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318417    5039 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318420    5039 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318423    5039 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318428    5039 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318430    5039 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318433    5039 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318436    5039 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318439    5039 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318442    5039 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318445    5039 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318448    5039 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318451    5039 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318455    5039 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318458    5039 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318461    5039 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318464    5039 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318467    5039 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318471    5039 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318477    5039 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318480    5039 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318483    5039 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:34:29.323853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318486    5039 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318488    5039 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318493    5039 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318496    5039 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318498    5039 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318502    5039 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318505    5039 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318508    5039 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318511    5039 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318514    5039 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318517    5039 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318520    5039 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318522    5039 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318526    5039 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318529    5039 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318532    5039 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318535    5039 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318538    5039 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318541    5039 flags.go:64] FLAG: --help="false"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318544    5039 flags.go:64] FLAG: --hostname-override="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318547    5039 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318550    5039 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318553    5039 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:34:29.324406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318557    5039 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318560    5039 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318588    5039 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318592    5039 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318595    5039 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318599    5039 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318602    5039 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318605    5039 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318607    5039 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318610    5039 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318613    5039 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318616    5039 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318619    5039 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318621    5039 flags.go:64] FLAG: --lock-file=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318624    5039 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318628    5039 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318631    5039 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318636    5039 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318639    5039 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318642    5039 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318645    5039 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318647    5039 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318651    5039 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318653    5039 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318656    5039 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:34:29.324991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318660    5039 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318663    5039 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318667    5039 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318669    5039 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318672    5039 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318675    5039 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318677    5039 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318680    5039 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318683    5039 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318686    5039 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318694    5039 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318697    5039 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318701    5039 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318704    5039 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318706    5039 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318710    5039 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318713    5039 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318715    5039 flags.go:64] FLAG: --port="10250"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318718    5039 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318721    5039 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0879c243918a54fd4"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318724    5039 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318727    5039 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318730    5039 flags.go:64] FLAG: --register-node="true"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318733    5039 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318739    5039 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318742    5039 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:34:29.325558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318744    5039 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318747    5039 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318750    5039 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318753    5039 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318756    5039 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318759    5039 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318762    5039 flags.go:64] FLAG: --runonce="false"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318764    5039 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318767    5039 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318770    5039 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318773    5039 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318775    5039 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318778    5039 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318781    5039 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318784    5039 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318787    5039 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318789    5039 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318794    5039 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318798    5039 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318801    5039 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318804    5039 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318807    5039 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318813    5039 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318816    5039 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318819    5039 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:34:29.326287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318823    5039 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318825    5039 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318828    5039 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318831    5039 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318834    5039 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318837    5039 flags.go:64] FLAG: --v="2"
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318841    5039 flags.go:64] FLAG: --version="false"
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318845    5039 flags.go:64] FLAG: --vmodule=""
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318849    5039 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.318853    5039 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318985    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318988    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318990    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318993    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318996    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.318998    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319001    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319003    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319006    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319008    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319010    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.326950 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319013    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319015    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319017    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319021    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319025    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319028    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319030    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319032    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319035    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319037    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319040    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319042    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319045    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319047    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319050    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319053    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319055    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319058    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319060    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319063    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.327442 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319065    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319068    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319070    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319072    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319075    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319077    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319080    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319082    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319084    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319087    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319089    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319091    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319094    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319097    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319099    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319103    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319107    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319109    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319112    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.327965 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319114    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319116    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319119    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319121    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319123    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319126    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319128    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319131    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319134    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319137    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319139    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319141    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319144    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319146    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319149    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319151    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319153    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319156    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319158    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319160    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.328605 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319163    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319165    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319167    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319170    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319172    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319175    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319177    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319180    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319183    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319187    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319189    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319191    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319194    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319196    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319199    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319201    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319203    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319206    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319209    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319213    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.329318 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319216    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319220    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319223    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319225    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319227    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319231    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319234    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319237    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319240    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319243    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319246    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319248    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319250    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319253    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319255    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.329868 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319257    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.319283    5039 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319318    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319322    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319326    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319331    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319334    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319338    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319341    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319343    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319345    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319348    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319350    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319353    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319355    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319357    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.330246 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319360    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319363    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319365    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319368    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319370    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319381    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319384    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319386    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319388    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319391    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319393    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319396    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319398    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319401    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319403    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319406    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319408    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319410    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319413    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319415    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.330676 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319418    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319422    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319424    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319428    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319430    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319433    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319435    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319438    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319440    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319443    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319446    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319448    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319450    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319454    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319456    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319459    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319461    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319464    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319467    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.331237 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319469    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319471    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319474    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319477    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319479    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319481    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319484    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319486    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319489    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319491    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319494    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319496    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319499    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319501    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319505    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319507    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319512    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319514    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319517    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319519    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.331806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319522    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319524    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319527    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319529    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319531    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319534    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319536    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319539    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319541    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319544    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319547    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319550    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319553    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319555    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319558    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319560    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319578    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319583    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319587    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319589    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.332287 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319592    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319594    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319597    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319600    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319604    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319606    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319611    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319614    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319617    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319619    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319623    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319626    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.319629    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.323359    5039 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.323381    5039 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.323399    5039 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.323403    5039 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323460    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323466    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.333008 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323469    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323472    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323474    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323478    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323481    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323484    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323486    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323489    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323491    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323494    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323496    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323499    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323501    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323503    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323506    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323508    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323512    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323514    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323517    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323520    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.333590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323522    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323525    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323527    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323529    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323532    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323534    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323537    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323539    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323542    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323544    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323546    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323549    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323551    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323553    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323556    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323558    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323561    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323580    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323583    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.334079 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323585    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323589    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323593    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323596    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323598    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323601    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323603    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323606    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323608    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323611    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323613    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323616    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323618    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323621    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323623    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323626    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323628    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323630    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323633    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.334518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323635    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323638    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323640    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323642    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323644    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323647    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323649    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323652    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323654    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323656    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323658    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323662    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323664    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323667    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323669    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323671    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323675    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323679    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323682    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.335078 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323684    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323687    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323689    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323692    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323694    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323697    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323699    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323702    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323704    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323707    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323709    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323711    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323714    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323717    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323719    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323721    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323724    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323726    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323728    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.335824 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323731    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323733    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323735    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323737    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323740    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323742    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323745    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323747    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.323773    5039 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323926    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323931    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323933    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323936    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323938    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323941    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323944    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.336399 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323946    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323949    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323951    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323954    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323956    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323959    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323961    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323964    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323966    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323968    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323971    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323973    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323975    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323978    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323981    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323983    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323985    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323988    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323990    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323993    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.336822 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.323997    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324000    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324003    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324006    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324008    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324011    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324013    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324015    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324018    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324020    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324022    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324025    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324027    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324030    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324032    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324034    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324039    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324042    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324044    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324047    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.337357 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324050    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324052    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324054    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324057    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324059    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324061    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324065    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324068    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324071    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324073    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324076    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324078    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324081    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324083    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324086    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324088    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324091    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324094    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324096    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.337986 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324099    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324101    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324104    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324106    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324108    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324111    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324113    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324116    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324118    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324123    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324126    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324129    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324131    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324133    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324136    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324138    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324141    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324143    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324146    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.338441 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324148    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324151    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324153    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324155    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324158    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324160    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324162    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324165    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324167    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324170    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324172    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324180    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324183    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324186    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324188    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324190    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324193    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324195    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324199    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.338978 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324202    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324205    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.324227    5039 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324279    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324284    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324288    5039 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324290    5039 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324293    5039 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324295    5039 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324298    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324300    5039 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324303    5039 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324305    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324307    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324310    5039 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324312    5039 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:29.339479 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324315    5039 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324317    5039 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324320    5039 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324322    5039 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324325    5039 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324327    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324330    5039 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324332    5039 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324334    5039 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324337    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324340    5039 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324342    5039 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324346    5039 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324349    5039 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324352    5039 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324356    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324359    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324361    5039 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324363    5039 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324366    5039 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:29.339934 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324369    5039 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324372    5039 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324374    5039 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324377    5039 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324379    5039 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324381    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324384    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324386    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324389    5039 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324391    5039 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324394    5039 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324396    5039 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324398    5039 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324401    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324403    5039 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324406    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324408    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324411    5039 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324413    5039 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:29.340552 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324417    5039 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324421    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324424    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324428    5039 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324431    5039 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324433    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324436    5039 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324438    5039 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324441    5039 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324444    5039 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324446    5039 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324448    5039 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324451    5039 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324454    5039 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324457    5039 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324460    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324463    5039 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324465    5039 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324468    5039 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324470    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:29.341160 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324473    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324476    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324479    5039 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324483    5039 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324486    5039 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324490    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324493    5039 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324497    5039 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324501    5039 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324505    5039 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324509    5039 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324512    5039 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324516    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324520    5039 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324523    5039 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324526    5039 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324530    5039 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324534    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324537    5039 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:29.341773 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324541    5039 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324545    5039 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324549    5039 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324553    5039 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324558    5039 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324562    5039 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324591    5039 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324596    5039 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324599    5039 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324603    5039 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324607    5039 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324610    5039 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324613    5039 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324616    5039 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:29.324619    5039 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.324673    5039 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.325685    5039 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.325738    5039 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 01:34:29.342361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.325919    5039 server.go:1014] "Starting client certificate rotation"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.326122    5039 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.326185    5039 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:28:49 +0000 UTC" deadline="2028-05-02 19:27:28.028132997 +0000 UTC"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.326266    5039 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13097h52m58.701870151s"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.326472    5039 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.327755    5039 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.329465    5039 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.331551    5039 log.go:25] "Validated CRI v1 image API"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.332443    5039 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:34:29.343071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.334541    5039 fs.go:135] Filesystem UUIDs: map[1b47b089-e7a6-40dc-a2c2-d502dce1ede9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a790cc8d-25aa-4c50-8fe4-bed5278eaf2c:/dev/nvme0n1p4]
Nov 04 01:34:29.343394 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.334560    5039 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1005ba1b09ed7ea8bf61b984c4badf72a4abb1abfac03100104c15d6e121bd27/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1005ba1b09ed7ea8bf61b984c4badf72a4abb1abfac03100104c15d6e121bd27/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/14760a4a284308dd6c057cec769f6709f61d2bb262e9d6f6cb62a6f838ee9a93/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/14760a4a284308dd6c057cec769f6709f61d2bb262e9d6f6cb62a6f838ee9a93/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/528629f5a4a1ed7cf0abb04404fb4ba7868511c0a1088ef695afeeaa28a2b164/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/528629f5a4a1ed7cf0abb04404fb4ba7868511c0a1088ef695afeeaa28a2b164/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b38800075d1133e8454aa1676444a67e422371d4de0e8c321910cbc182fb4431/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b38800075d1133e8454aa1676444a67e422371d4de0e8c321910cbc182fb4431/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c24fe813338fab33377a99670a97eae5bca619cd8a19fd00efb058cb80cc38f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c24fe813338fab33377a99670a97eae5bca619cd8a19fd00efb058cb80cc38f7/userdata/shm major:0 minor:334 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a/userdata/shm major:0 minor:328 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dff8563496a83adbf36169bd514839a31531f5745541706700246c18c7e87a4e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dff8563496a83adbf36169bd514839a31531f5745541706700246c18c7e87a4e/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f7dd48e88a5e0b780d6de985441712a8e0ef3f18bac3c9224c2b7194c900b5b0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f7dd48e88a5e0b780d6de985441712a8e0ef3f18bac3c9224c2b7194c900b5b0/userdata/shm major:0 minor:362 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/016762ac-ad73-46b0-8059-9b4b2b670043/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/016762ac-ad73-46b0-8059-9b4b2b670043/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~projected/kube-api-access-tw5xq:{mountpoint:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~projected/kube-api-access-tw5xq major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~projected/kube-api-access-4pzgb:{mountpoint:/var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~projected/kube-api-access-4pzgb major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~secret/default-certificate major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~secret/stats-auth major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/081f052e-3e74-4fd5-8d12-d91aa017a999/volumes/kubernetes.io~projected/kube-api-access-6mc9h:{mountpoint:/var/lib/kubelet/pods/081f052e-3e74-4fd5-8d12-d91aa017a999/volumes/kubernetes.io~projected/kube-api-access-6mc9h major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/081f052e-3e74-4fd5-8d12-d91aa017a999/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/081f052e-3e74-4fd5-8d12-d91aa017a999/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10bc58fa-af70-41c2-9e8c-0cf7774b67fe/volumes/kubernetes.io~projected/kube-api-access-wl9jp:{mountpoint:/var/lib/kubelet/pods/10bc58fa-af70-41c2-9e8c-0cf7774b67fe/volumes/kubernetes.io~projected/kube-api-access-wl9jp major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4164bd42-81bf-478d-8409-001aed0783dc/volumes/kubernetes.io~projected/kube-api-access-dpwrx:{mountpoint:/var/lib/kubelet/pods/4164bd42-81bf-478d-8409-001aed0783dc/volumes/kubernetes.io~projected/kube-api-access-dpwrx major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6817a7e5-506a-4ae2-8128-5b6601281286/volumes/kubernetes.io~projected/kube-api-access-zq6zm:{mountpoint:/var/lib/kubelet/pods/6817a7e5-506a-4ae2-8128-5b6601281286/volumes/kubernetes.io~projected/kube-api-access-zq6zm major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69b134d1-aa78-45f4-a572-7bc4171c5a56/volumes/kubernetes.io~projected/kube-api-access-zcsww:{mountpoint:/var/lib/kubelet/pods/69b134d1-aa78-45f4-a572-7bc4171c5a56/volumes/kubernetes.io~projected/kube-api-access-zcsww major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a1464fb-a344-4bd9-8406-6fe464ff4fa0/volumes/kubernetes.io~projected/kube-api-access-l4d5f:{mountpoint:/var/lib/kubelet/pods/6a1464fb-a344-4bd9-8406-6fe464ff4fa0/volumes/kubernetes.io~projected/kube-api-access-l4d5f major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76b11319-689c-4ba5-83ec-afe254b80174/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/76b11319-689c-4ba5-83ec-afe254b80174/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea/volumes/kubernetes.io~projected/kube-api-access-4tld9:{mountpoint:/var/lib/kubelet/pods/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea/volumes/kubernetes.io~projected/kube-api-access-4tld9 major:0 minor:331 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81cc5d87-5a2f-43e5-ba84-04993c15476e/volumes/kubernetes.io~projected/kube-api-access-2qvxs:{mountpoint:/var/lib/kubelet/pods/81cc5d87-5a2f-43e5-ba84-04993c15476e/volumes/kubernetes.io~projected/kube-api-access-2qvxs major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ec90134-2d9f-418f-98e5-877305e1a0b2/volumes/kubernetes.io~projected/kube-api-access-q9km5:{mountpoint:/var/lib/kubelet/pods/8ec90134-2d9f-418f-98e5-877305e1a0b2/volumes/kubernetes.io~projected/kube-api-access-q9km5 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/909efb32-95fc-4b85-bf58-949637c49c16/volumes/kubernetes.io~projected/kube-api-access-jcbz6:{mountpoint:/var/lib/kubelet/pods/909efb32-95fc-4b85-bf58-949637c49c16/volumes/kubernetes.io~projected/kube-api-access-jcbz6 major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/928e6807-f385-42a2-89d5-e9ed3b1089cf/volumes/kubernetes.io~projected/kube-api-access-47p5l:{mountpoint:/var/lib/kubelet/pods/928e6807-f385-42a2-89d5-e9ed3b1089cf/volumes/kubernetes.io~projected/kube-api-access-47p5l major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9377de09-4111-4499-a1d7-62e407f76422/volumes/kubernetes.io~projected/kube-api-access-hp9v2:{mountpoint:/var/lib/kubelet/pods/9377de09-4111-4499-a1d7-62e407f76422/volumes/kubernetes.io~projected/kube-api-access-hp9v2 major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9377de09-4111-4499-a1d7-62e407f76422/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/9377de09-4111-4499-a1d7-62e407f76422/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b16b031-58ec-4d17-82b1-61a7c952c887/volumes/kubernetes.io~projected/kube-api-access-w947m:{mountpoint:/var/lib/kubelet/pods/9b16b031-58ec-4d17-82b1-61a7c952c887/volumes/kubernetes.io~projected/kube-api-access-w947m major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5bea9b9-e510-4b8b-bc95-5ad38581b40c/volumes/kubernetes.io~projected/kube-api-access-fq5xp:{mountpoint:/var/lib/kubelet/pods/a5bea9b9-e510-4b8b-bc95-5ad38581b40c/volumes/kubernetes.io~projected/kube-api-access-fq5xp major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b546556a-5219-406b-88d0-a3b490c7b259/volumes/kubernetes.io~projected/kube-api-access-87w7x:{mountpoint:/var/lib/kubelet/pods/b546556a-5219-406b-88d0-a3b490c7b259/volumes/kubernetes.io~projected/kube-api-access-87w7x major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b546556a-5219-406b-88d0-a3b490c7b259/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/b546556a-5219-406b-88d0-a3b490c7b259/volumes/kubernetes.io~secret/serving-cert major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd5494d4-4343-4f18-b981-b39fe6a54034/volumes/kubernetes.io~projected/kube-api-access-mw9md:{mountpoint:/var/lib/kubelet/pods/bd5494d4-4343-4f18-b981-b39fe6a54034/volumes/kubernetes.io~projected/kube-api-access-mw9md major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd5494d4-4343-4f18-b981-b39fe6a54034/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/bd5494d4-4343-4f18-b981-b39fe6a54034/volumes/kubernetes.io~secret/signing-key major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0/volumes/kubernetes.io~projected/kube-api-access-j7rst:{mountpoint:/var/lib/kubelet/pods/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0/volumes/kubernetes.io~projected/kube-api-access-j7rst major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~empty-dir/tmp major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~projected/kube-api-access-mmsgm:{mountpoint:/var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~projected/kube-api-access-mmsgm major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c65a7a37-b3e1-46a1-9ed7-204de523fdc7/volumes/kubernetes.io~projected/kube-api-access-9sngl:{mountpoint:/var/lib/kubelet/pods/c65a7a37-b3e1-46a1-9ed7-204de523fdc7/volumes/kubernetes.io~projected/kube-api-access-9sngl major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c7411ec4-7d75-44ad-ae7c-fae47eee36f4/volumes/kubernetes.io~projected/kube-api-access-qp9mj:{mountpoint:/var/lib/kubelet/pods/c7411ec4-7d75-44ad-ae7c-fae47eee36f4/volumes/kubernetes.io~projected/kube-api-access-qp9mj major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~projected/kube-api-access-ptfwf:{mountpoint:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~projected/kube-api-access-ptfwf major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfbd09a6-31fc-4e11-bf84-da0864ec84ff/volumes/kubernetes.io~projected/kube-api-access-f8dml:{mountpoint:/var/lib/kubelet/pods/cfbd09a6-31fc-4e11-bf84-da0864ec84ff/volumes/kubernetes.io~projected/kube-api-access-f8dml major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8/volumes/kubernetes.io~projected/kube-api-access-j5z6n:{mountpoint:/var/lib/kubelet/pods/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8/volumes/kubernetes.io~projected/kube-api-access-j5z6n major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4f41073-2c64-44f1-9153-ca4e0e2c0b50/volumes/kubernetes.io~projected/kube-api-access-74628:{mountpoint:/var/lib/kubelet/pods/d4f41073-2c64-44f1-9153-ca4e0e2c0b50/volumes/kubernetes.io~projected/kube-api-access-74628 major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4f41073-2c64-44f1-9153-ca4e0e2c0b50/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d4f41073-2c64-44f1-9153-ca4e0e2c0b50/volumes/kubernetes.io~secret/cert major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5017035-43de-4aec-bcb7-cbdee8be6bc3/volumes/kubernetes.io~projected/kube-api-access-692kd:{mountpoint:/var/lib/kubelet/pods/d5017035-43de-4aec-bcb7-cbdee8be6bc3/volumes/kubernetes.io~projected/kube-api-access-692kd major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df3c83fd-42b4-423d-a2a2-9da642a21118/volumes/kubernetes.io~projected/kube-api-access-nr76c:{mountpoint:/var/lib/kubelet/pods/df3c83fd-42b4-423d-a2a2-9da642a21118/volumes/kubernetes.io~projected/kube-api-access-nr76c major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df3c83fd-42b4-423d-a2a2-9da642a21118/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/df3c83fd-42b4-423d-a2a2-9da642a21118/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e13536d8-6703-46a7-9e8e-8136e1ff8979/volumes/kubernetes.io~projected/kube-api-access-fr9hr:{mountpoint:/var/lib/kubelet/pods/e13536d8-6703-46a7-9e8e-8136e1ff8979/volumes/kubernetes.io~projected/kube-api-access-fr9hr major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volumes/kubernetes.io~projected/kube-api-access-5j4hz:{mountpoint:/var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volumes/kubernetes.io~projected/kube-api-access-5j4hz major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db/volumes/kubernetes.io~projected/kube-api-access-67jjd:{mountpoint:/var/lib/kubelet/pods/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db/volumes/kubernetes.io~projected/kube-api-access-67jjd major:0 minor:212 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/b2d4ddeacfbdb22a1ab1af285d558f6269decc6904e05279a3f23561bfcc7d85/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/2a7d15f5b2c9a518ef479d943cd3e3b66b9029f16a9d98c1dbab6bbaa68b81b8/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/574af503917d1a312c5d16077181ad000e7c6c7dc50f96b5f0a7af76a6c78949/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/d58d47d02651edfa1bfb03a2c42bf8c28854999aacb9173cc7f7e51af259c613/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/4188536b58b9b2cd8978f21aa1296f297b3ed3dfb235138db9742737b43279fb/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/be308ef3e62d05a6e8a24be65e325fa3e8538867b57eed1f20f755c6fb049a7a/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/3ad00a7c7fd46a1003f065db33140108c82f925df05c9cbb83608e8795093c06/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/422575c1442975c2f315581b3074da238545856faf881bd483ccc23865cbed02/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/9bae2a9d95076d5c04701a977f02d8ff4252b3e72b0961cd36fa9a1b7b1d7fbf/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/a63b000efc6a4b6a9aeb8eb1234c6f4d7d9ec090f674ecc22b9f2b152890ffb0/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/6b2a7763012c0290822da2167d66dbc1e48a77f6db69d02568e0b6fbe2d2e421/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/d5e4ad45ec22edc0ea731c296856d4cab66bbb86fc106fed798dad0e158eadc2/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/5e726a2cdb700db7cef094b21dfecdf1632aec22ab6188032a2d8795419e9368/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/6b367170eb7c754f1135a8894c8d4cb6829800301239e6e6d9de9f81178f3c30/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/9de3dba0e3384aeffc5b9e1986350dba5c40a54910c69587c229247d4b8d36c3/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/be033dc63b2fd1bec2ba3a6253686f2889fd1550996d63bb83c8d0660a5432a0/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/89b930e18c1dfec4c6ec8f2b92c3553c33c05cee74bdddb65db981f7a36ce7d2/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/4e20d3f3dde00617e30303ce3fb3edd5c435933a4e80fd341e09a56b0d9cfe9e/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/671776c409cc7a5ed17dc4acae265b995c9c36b538a7d68eeac5cb52bfc0cfc6/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/6488a73450dcffc09ffaa1be1bdd662019b988ca2c3a8df9eac98687968f5818/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-188:{mountpoint:/var/lib/containers/storage/overlay/19a68bcf876e9cc1d23eeb67f77444880b1868a2080d74b6f1275f3274008cc4/merged major:0 minor:188 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/add09ae6477d32a2f8edea792a8a5167321b2d5ad0121409fbb28b31276f6282/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/f51e087f34765037496f21224ee51faca0febcc5a4e58a2b24503f63b7ee31cb/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/0ec5c1f9507d9b0a47d0d11806bb7ff75fd342ee6677852e9e7d0414a692a1f3/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/09bcff5ce3e8b3a09f419c8bae9fddc6a7b29946c7c05c92df46656d79a9313e/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/056446bec085815580d4088afd84d269bda76eeeb336093acda8b6b3312312a7/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/ab2cd622e5071b393c11030d43c47579a449346e26b729e0da350bc6b8bcb31f/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/55830d0715c31320e47a5a237e1caadf37be2473c1811be149d3f862090b2e14/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/e5ae6f8d1353cc3f6b8bada56e24ac63dc068141eceaa5da76a24d3b20f4f3f7/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/931e9daf37e0876097153612be3496819e93fe2cbd61de6a48ab9aa37ad98f73/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/eebb35f5e08276f4bf3a844bf897632fb7c63d0e1284f4bb43ae899facea3539/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/0348f3f41857b81a200429b68bab4d5db23159fdfb176eb9b675632d68baebc7/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/306782d804ccf370c4845135df7d0330a848d9a563f2ba22e58d2f7496384218/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/12d24799f81be845e9cbac5b32fd2388bc48cbda7099f0aa4166849b14a5fc78/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/cd029f187f397d4247a7f48b40692a97203cd33eb12e4c5034f748665cd7bb45/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/3cb3f6bae531f0a7287a7046770e1380ed40ad8d1b36e6da8685930f761e95c9/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/c842a38cfbba22783a4fb43bf74fcbed58497d89b904819a0bf47e011dee0a66/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/b1829b6a7136d7f07b273e049ced44913ddcce05cf97d6be7d69102eb93c8cb7/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/45e29a84922fe2bd022e7fea17b101851ee032240680ab5c89d52d2f3ce46f8e/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/5dcaf9b2c2673ba0b48754984890c29f63182a26acf08f9b92f2278f28f3ca21/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/3aeeff6dff0e3c87d6c9e51a7a291d367fb5fad20bf135f86c36d75d77aa526c/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/857de9c621df43375e1bda8558fbb729de27837a295cbdab76ff8add383b1c87/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/10277a03a521fbd14413e7ab0c1d37e852ee6a639a36b262d6036b3cb4f52b77/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/514a5aae39afbb6a004612b548765bbbbc1c479a9a8e4ee77fa3851e24bccb0e/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/846e393038f1d24d8057692069f8e45cc4fb7935bc53fd03c25ff9946e4c0a66/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/609a380c84357ff7b68af8fb75ece5ec48b9b9db8cb56852ceecb9e98207464b/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/114215335a4e746159d6112a0c7601a32b29e0eceee2bbd3e55de1e8a19982bf/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/f6e5486fcc940c780833f0802ed11b8a4272a51eb390128d1909c0cf4a25b9f9/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/ff42cbb92bd9e3fd8aebb1c8a2feb668767b126bac89eb0028e9838319bed342/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/f8e4cd1287ad0c0be39fde4ebdcaabde016e5cb349b264a34edd435e592c718e/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/0e5b7a6ad8558ef11b2235d1bed1435bb0aeae5249c2bd1c029288c7783784d6/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/6a61e97d29a7dc82ac6b25033828d55f288f0d0ba3642491232aff5679c8c5d7/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/dc6b0f5553161dbda978077764e86003eed7302efcfdf13ed817945b285dd3c1/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/1a8518c63ecd9e86a101a0dc659d997d2cf19f144bfd745ced62f5000e242294/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/645d1ab33e9677a49afa95122bb82194d0d0475e8163968a5c9ab2dc47fc7a76/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/61fef2f69370e9536bd8950946a6104360f1d9c866b54095ed7b125f6b92cb80/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/4928c22f8713ef17e96dd9a41cb9800d2f9f966f63946e378a48936fb33b954d/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/2e55b6acef9ada3782b5e15e5c49fd5f48003b6960267e096a72f64ed48883b7/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/ee373a7b03d94de61b1f5f79b27b89716aa882f8727cb7876cd8b5206a08617c/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/95129afb9fac303ad72ff414a33073df71f47fe062c1e0371ea93d68b299bd09/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/1b79ab34152df7f694817fa4c6527818efa93f15718df77daa0eecb5d48d94cc/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/e015809c4f16394572a3b2e51cf2f97741a8f2864c3b3a9b073c8193d9d9afa5/merged major:0 minor:98 fsType:overlay blockSize:0}]
Nov 04 01:34:29.344202 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.343798    5039 manager.go:217] Machine: {Timestamp:2026-11-04 01:34:29.343308441 +0000 UTC m=+0.059495287 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3201695 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25a3dd573d45a17c348650a16ffb53 SystemUUID:ec25a3dd-573d-45a1-7c34-8650a16ffb53 BootID:f5b3140f-d33f-431d-b311-564fa6426591 Filesystems:[{Device:/var/lib/kubelet/pods/9b16b031-58ec-4d17-82b1-61a7c952c887/volumes/kubernetes.io~projected/kube-api-access-w947m DeviceMajor:0 DeviceMinor:235 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1005ba1b09ed7ea8bf61b984c4badf72a4abb1abfac03100104c15d6e121bd27/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b546556a-5219-406b-88d0-a3b490c7b259/volumes/kubernetes.io~projected/kube-api-access-87w7x DeviceMajor:0 DeviceMinor:234 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8b2e0745762be0ff36d7cd843bc0fb7039fd172df364e37be1caf2b2de72fd8b/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d4f41073-2c64-44f1-9153-ca4e0e2c0b50/volumes/kubernetes.io~projected/kube-api-access-74628 DeviceMajor:0 DeviceMinor:233 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:209 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/df3c83fd-42b4-423d-a2a2-9da642a21118/volumes/kubernetes.io~projected/kube-api-access-nr76c DeviceMajor:0 DeviceMinor:221 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bd5494d4-4343-4f18-b981-b39fe6a54034/volumes/kubernetes.io~projected/kube-api-access-mw9md DeviceMajor:0 DeviceMinor:361 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b440ae00f123d49743dde39e41d07fec1149f88e407061d228db8f97d2f61ea3/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cfbd09a6-31fc-4e11-bf84-da0864ec84ff/volumes/kubernetes.io~projected/kube-api-access-f8dml DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~projected/kube-api-access-4pzgb DeviceMajor:0 DeviceMinor:208 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8/volumes/kubernetes.io~projected/kube-api-access-j5z6n DeviceMajor:0 DeviceMinor:232 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-188 DeviceMajor:0 DeviceMinor:188 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db/volumes/kubernetes.io~projected/kube-api-access-67jjd DeviceMajor:0 DeviceMinor:212 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c65a7a37-b3e1-46a1-9ed7-204de523fdc7/volumes/kubernetes.io~projected/kube-api-access-9sngl DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~projected/kube-api-access-tw5xq DeviceMajor:0 DeviceMinor:213 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a/userdata/shm DeviceMajor:0 DeviceMinor:328 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54e83d025dcea834dcf3430c16a4502a3467a1cb2b0621d18848214d2177c63b/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc15959f5b0a19b9fddbb729952eac6999a61d3c17028a26226c28799c94df6d/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0/volumes/kubernetes.io~projected/kube-api-access-j7rst DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4164bd42-81bf-478d-8409-001aed0783dc/volumes/kubernetes.io~projected/kube-api-access-dpwrx DeviceMajor:0 DeviceMinor:322 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1850eec52a807e0f86ce435c88f27c031d2a6d7d92437112cc9c2ba01d8afb7c/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a5bea9b9-e510-4b8b-bc95-5ad38581b40c/volumes/kubernetes.io~projected/kube-api-access-fq5xp DeviceMajor:0 DeviceMinor:288 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/53a3d4040996d920bab7187aef0cb9e2df2afb359225d1856396756034d86298/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea/volumes/kubernetes.io~projected/kube-api-access-4tld9 DeviceMajor:0 DeviceMinor:331 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/df3c83fd-42b4-423d-a2a2-9da642a21118/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f7dd48e88a5e0b780d6de985441712a8e0ef3f18bac3c9224c2b7194c900b5b0/userdata/shm DeviceMajor:0 DeviceMinor:362 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2418338cea7dd9fb6cb5916ddbaa6dc7a6c666463c75455b6a0228017ef535e9/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c92578f8f6965d6828739489baea898ab2c4e25444045344a11b7e70220eb67e/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/69b134d1-aa78-45f4-a572-7bc4171c5a56/volumes/kubernetes.io~projected/kube-api-access-zcsww DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6a1464fb-a344-4bd9-8406-6fe464ff4fa0/volumes/kubernetes.io~projected/kube-api-access-l4d5f DeviceMajor:0 DeviceMinor:218 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2cc3ba7504e0802e6226fe4463e8d72ba74e27dea696c1b55829ede396d84109/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9377de09-4111-4499-a1d7-62e407f76422/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:219 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/df2d2609470774ea5949becc4a5089e02135c8155d7fd5a1a4025c69abda3190/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd5494d4-4343-4f18-b981-b39fe6a54034/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:356 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c04efb2b-37d6-4ded-a359-ccb48c497b43/volumes/kubernetes.io~projected/kube-api-access-mmsgm DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:205 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~projected/kube-api-access-ptfwf DeviceMajor:0 DeviceMinor:216 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b546556a-5219-406b-88d0-a3b490c7b259/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:222 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e3e26cb512279e8546e526b59d3d3578e38282d19f14a6da732030a4f17090bc/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce0c59de3794322b6862ef7040bdd80bb4d10f78078926556416bbd1db2296ad/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8ec90134-2d9f-418f-98e5-877305e1a0b2/volumes/kubernetes.io~projected/kube-api-access-q9km5 DeviceMajor:0 DeviceMinor:214 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9377de09-4111-4499-a1d7-62e407f76422/volumes/kubernetes.io~projected/kube-api-access-hp9v2 DeviceMajor:0 DeviceMinor:215 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/909efb32-95fc-4b85-bf58-949637c49c16/volumes/kubernetes.io~projected/kube-api-access-jcbz6 DeviceMajor:0 DeviceMinor:323 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/016762ac-ad73-46b0-8059-9b4b2b670043/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81cc5d87-5a2f-43e5-ba84-04993c15476e/volumes/kubernetes.io~projected/kube-api-access-2qvxs DeviceMajor:0 DeviceMinor:217 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e13536d8-6703-46a7-9e8e-8136e1ff8979/volumes/kubernetes.io~projected/kube-api-access-fr9hr DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b985fb65da07b13c9bd8d9948a828afada50a3532321312922810fb257d2b89f/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/10bc58fa-af70-41c2-9e8c-0cf7774b67fe/volumes/kubernetes.io~projected/kube-api-access-wl9jp DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c24fe813338fab33377a99670a97eae5bca619cd8a19fd00efb058cb80cc38f7/userdata/shm DeviceMajor:0 DeviceMinor:334 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b38800075d1133e8454aa1676444a67e422371d4de0e8c321910cbc182fb4431/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6817a7e5-506a-4ae2-8128-5b6601281286/volumes/kubernetes.io~projected/kube-api-access-zq6zm DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/483ec086c0e9f37c18f73aa7bc89860b38b46764c8b8e09d52766d51a4b9475e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/07ebf665-177c-48e7-969c-28057a4ff1a4/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/081f052e-3e74-4fd5-8d12-d91aa017a999/volumes/kubernetes.io~projected/kube-api-access-6mc9h DeviceMajor:0 DeviceMinor:220 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/14760a4a284308dd6c057cec769f6709f61d2bb262e9d6f6cb62a6f838ee9a93/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a27a1ce446bba9b353ad3d6870461ce6b9c928db22a8e3a0af6025e473acfeb3/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c7411ec4-7d75-44ad-ae7c-fae47eee36f4/volumes/kubernetes.io~projected/kube-api-access-qp9mj DeviceMajor:0 DeviceMinor:210 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/528629f5a4a1ed7cf0abb04404fb4ba7868511c0a1088ef695afeeaa28a2b164/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dff8563496a83adbf36169bd514839a31531f5745541706700246c18c7e87a4e/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6d79befb102d827cdc0590da44f1262f0856e83ca396d3c83d0c3935ebf254bd/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d5017035-43de-4aec-bcb7-cbdee8be6bc3/volumes/kubernetes.io~projected/kube-api-access-692kd DeviceMajor:0 DeviceMinor:330 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fcb074c6-4262-4601-9730-f4ae23ee938b/volumes/kubernetes.io~projected/kube-api-access-5j4hz DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/928e6807-f385-42a2-89d5-e9ed3b1089cf/volumes/kubernetes.io~projected/kube-api-access-47p5l DeviceMajor:0 DeviceMinor:211 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d4f41073-2c64-44f1-9153-ca4e0e2c0b50/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:223 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76b11319-689c-4ba5-83ec-afe254b80174/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:293 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/081f052e-3e74-4fd5-8d12-d91aa017a999/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:14760a4a284308d MacAddress:5a:5b:a4:e5:ff:56 Speed:10000 Mtu:8901} {Name:1850eec52a807e0 MacAddress:c2:99:1b:89:64:93 Speed:10000 Mtu:8901} {Name:2418338cea7dd9f MacAddress:de:b1:21:97:fc:fa Speed:10000 Mtu:8901} {Name:528629f5a4a1ed7 MacAddress:2a:26:6a:6a:51:8a Speed:10000 Mtu:8901} {Name:6d79befb102d827 MacAddress:3a:c5:9d:f2:4a:82 Speed:10000 Mtu:8901} {Name:a27a1ce446bba9b MacAddress:d6:42:f2:30:b1:f0 Speed:10000 Mtu:8901} {Name:b38800075d1133e MacAddress:9e:25:18:72:5b:56 Speed:10000 Mtu:8901} {Name:b440ae00f123d49 MacAddress:0e:9b:4f:ec:10:c9 Speed:10000 Mtu:8901} {Name:b985fb65da07b13 MacAddress:4e:7e:d0:67:72:cb Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:72:5b:4b:f8:3b Speed:0 Mtu:9001} {Name:br-int MacAddress:2e:0c:21:e6:7c:50 Speed:0 Mtu:8901} {Name:c24fe813338fab3 MacAddress:12:65:e3:40:a6:48 Speed:10000 Mtu:8901} {Name:c92578f8f6965d6 MacAddress:2a:be:22:6c:ef:0d Speed:10000 Mtu:8901} {Name:de5b95e01eded23 MacAddress:12:47:9b:fb:04:cf Speed:10000 Mtu:8901} {Name:df2d2609470774e MacAddress:ea:64:43:f9:eb:e1 Speed:10000 Mtu:8901} {Name:dff8563496a83ad MacAddress:72:c3:bf:b9:b8:28 Speed:10000 Mtu:8901} {Name:e3e26cb512279e8 MacAddress:56:74:70:98:3e:86 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:72:5b:4b:f8:3b Speed:0 Mtu:9001} {Name:f7dd48e88a5e0b7 MacAddress:96:c1:44:48:6d:73 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:46:b5:7c:8a:03:14 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:9e:7c:3c:22:7b:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:34:29.344388 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344207    5039 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:34:29.344388 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344294    5039 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:34:29.344651 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344628    5039 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:34:29.344782 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344652    5039 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-18.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:34:29.344823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344785    5039 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:34:29.344823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344793    5039 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:34:29.344823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344803    5039 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:34:29.344823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344822    5039 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:34:29.344926 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.344908    5039 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:34:29.345111 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345102    5039 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:34:29.345214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345206    5039 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:34:29.345244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345239    5039 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:34:29.345271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345252    5039 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:34:29.345271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345262    5039 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:34:29.345271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345271    5039 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:34:29.345350 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.345279    5039 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:34:29.346093 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.346081    5039 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:34:29.346093 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.346093    5039 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:34:29.346527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.346508    5039 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:34:29.348630 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.348618    5039 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:34:29.348676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.348630    5039 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:34:29.348737 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.348728    5039 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 01:34:29.348922 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.348911    5039 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:34:29.348954 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.348940    5039 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:34:29.349034 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349025    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:34:29.349062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349044    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:34:29.349062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349057    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:34:29.349117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349068    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:34:29.349117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349084    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:34:29.349117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349107    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:34:29.349195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349123    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:34:29.349195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349137    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:34:29.349195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349154    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:34:29.349195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349162    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:34:29.349195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349189    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:34:29.349437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349202    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:34:29.349437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349231    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:34:29.349437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.349240    5039 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:34:29.353389 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.353374    5039 server.go:1263] "Started kubelet"
Nov 04 01:34:29.357151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.353883    5039 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:34:29.357151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.353974    5039 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:34:29.357151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.353925    5039 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:34:29.354038 ip-10-0-140-18 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:34:29.358119 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.357337    5039 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.358996    5039 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.363118    5039 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.363383    5039 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.363476    5039 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:28:50 +0000 UTC" deadline="2028-06-15 06:06:35.04287783 +0000 UTC"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.363515    5039 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14140h32m5.679366806s"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.363550    5039 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:34:29.364223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.363558    5039 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:34:29.364603 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.364316    5039 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.365261    5039 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.366282    5039 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.366299    5039 factory.go:55] Registering systemd factory
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.366308    5039 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367732    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367776    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" volumeName="kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367790    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69b134d1-aa78-45f4-a572-7bc4171c5a56" volumeName="kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-daemon-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367802    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9377de09-4111-4499-a1d7-62e407f76422" volumeName="kubernetes.io/secret/9377de09-4111-4499-a1d7-62e407f76422-serving-cert" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367814    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b546556a-5219-406b-88d0-a3b490c7b259" volumeName="kubernetes.io/secret/b546556a-5219-406b-88d0-a3b490c7b259-serving-cert" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367825    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367836    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" volumeName="kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367853    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3b79cfe-4f1c-4786-9f97-e87ce36cbba8" volumeName="kubernetes.io/empty-dir/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-tmp-dir" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367866    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367877    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69b134d1-aa78-45f4-a572-7bc4171c5a56" volumeName="kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367889    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b546556a-5219-406b-88d0-a3b490c7b259" volumeName="kubernetes.io/projected/b546556a-5219-406b-88d0-a3b490c7b259-kube-api-access-87w7x" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367899    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e13536d8-6703-46a7-9e8e-8136e1ff8979" volumeName="kubernetes.io/empty-dir/e13536d8-6703-46a7-9e8e-8136e1ff8979-tmp-dir" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367918    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a1464fb-a344-4bd9-8406-6fe464ff4fa0" volumeName="kubernetes.io/projected/6a1464fb-a344-4bd9-8406-6fe464ff4fa0-kube-api-access-l4d5f" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367930    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9377de09-4111-4499-a1d7-62e407f76422" volumeName="kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367943    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="be3ebfff-ac2e-4429-ac1e-1fb87d7599d0" volumeName="kubernetes.io/configmap/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-serviceca" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367960    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c04efb2b-37d6-4ded-a359-ccb48c497b43" volumeName="kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-tuned" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367971    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367981    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df3c83fd-42b4-423d-a2a2-9da642a21118" volumeName="kubernetes.io/secret/df3c83fd-42b4-423d-a2a2-9da642a21118-serving-cert" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.367999    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07ebf665-177c-48e7-969c-28057a4ff1a4" volumeName="kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-stats-auth" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368009    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76b11319-689c-4ba5-83ec-afe254b80174" volumeName="kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368021    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="928e6807-f385-42a2-89d5-e9ed3b1089cf" volumeName="kubernetes.io/projected/928e6807-f385-42a2-89d5-e9ed3b1089cf-kube-api-access-47p5l" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368031    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368043    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3b79cfe-4f1c-4786-9f97-e87ce36cbba8" volumeName="kubernetes.io/projected/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-kube-api-access-j5z6n" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368049    5039 factory.go:153] Registering CRI-O factory
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368056    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db" volumeName="kubernetes.io/projected/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db-kube-api-access-67jjd" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368067    5039 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368069    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="081f052e-3e74-4fd5-8d12-d91aa017a999" volumeName="kubernetes.io/projected/081f052e-3e74-4fd5-8d12-d91aa017a999-kube-api-access-6mc9h" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368080    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7896c15f-392c-49fb-8d4c-9fdbbc3de6ea" volumeName="kubernetes.io/projected/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea-kube-api-access-4tld9" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368093    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368105    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07ebf665-177c-48e7-969c-28057a4ff1a4" volumeName="kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-default-certificate" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368121    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4164bd42-81bf-478d-8409-001aed0783dc" volumeName="kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368143    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368164    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3b79cfe-4f1c-4786-9f97-e87ce36cbba8" volumeName="kubernetes.io/configmap/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-config-volume" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368187    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcb074c6-4262-4601-9730-f4ae23ee938b" volumeName="kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-env-overrides" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368198    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="081f052e-3e74-4fd5-8d12-d91aa017a999" volumeName="kubernetes.io/secret/081f052e-3e74-4fd5-8d12-d91aa017a999-serving-cert" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368209    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909efb32-95fc-4b85-bf58-949637c49c16" volumeName="kubernetes.io/empty-dir/909efb32-95fc-4b85-bf58-949637c49c16-data-volume" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368222    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="909efb32-95fc-4b85-bf58-949637c49c16" volumeName="kubernetes.io/projected/909efb32-95fc-4b85-bf58-949637c49c16-kube-api-access-jcbz6" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368232    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b16b031-58ec-4d17-82b1-61a7c952c887" volumeName="kubernetes.io/projected/9b16b031-58ec-4d17-82b1-61a7c952c887-kube-api-access-w947m" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368244    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd5494d4-4343-4f18-b981-b39fe6a54034" volumeName="kubernetes.io/projected/bd5494d4-4343-4f18-b981-b39fe6a54034-kube-api-access-mw9md" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368256    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcb074c6-4262-4601-9730-f4ae23ee938b" volumeName="kubernetes.io/secret/fcb074c6-4262-4601-9730-f4ae23ee938b-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368268    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368281    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368293    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9377de09-4111-4499-a1d7-62e407f76422" volumeName="kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-snapshots" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368304    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c04efb2b-37d6-4ded-a359-ccb48c497b43" volumeName="kubernetes.io/projected/c04efb2b-37d6-4ded-a359-ccb48c497b43-kube-api-access-mmsgm" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368315    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" volumeName="kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368334    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df3c83fd-42b4-423d-a2a2-9da642a21118" volumeName="kubernetes.io/configmap/df3c83fd-42b4-423d-a2a2-9da642a21118-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368344    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368356    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07ebf665-177c-48e7-969c-28057a4ff1a4" volumeName="kubernetes.io/projected/07ebf665-177c-48e7-969c-28057a4ff1a4-kube-api-access-4pzgb" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368375    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6817a7e5-506a-4ae2-8128-5b6601281286" volumeName="kubernetes.io/configmap/6817a7e5-506a-4ae2-8128-5b6601281286-iptables-alerter-script" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368387    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5bea9b9-e510-4b8b-bc95-5ad38581b40c" volumeName="kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368397    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b546556a-5219-406b-88d0-a3b490c7b259" volumeName="kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368410    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="016762ac-ad73-46b0-8059-9b4b2b670043" volumeName="kubernetes.io/configmap/016762ac-ad73-46b0-8059-9b4b2b670043-konnectivity-ca" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368421    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="016762ac-ad73-46b0-8059-9b4b2b670043" volumeName="kubernetes.io/secret/016762ac-ad73-46b0-8059-9b4b2b670043-agent-certs" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368432    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="081f052e-3e74-4fd5-8d12-d91aa017a999" volumeName="kubernetes.io/configmap/081f052e-3e74-4fd5-8d12-d91aa017a999-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368445    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10bc58fa-af70-41c2-9e8c-0cf7774b67fe" volumeName="kubernetes.io/projected/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-kube-api-access-wl9jp" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368456    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6817a7e5-506a-4ae2-8128-5b6601281286" volumeName="kubernetes.io/projected/6817a7e5-506a-4ae2-8128-5b6601281286-kube-api-access-zq6zm" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368467    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4f41073-2c64-44f1-9153-ca4e0e2c0b50" volumeName="kubernetes.io/projected/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-kube-api-access-74628" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368479    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4f41073-2c64-44f1-9153-ca4e0e2c0b50" volumeName="kubernetes.io/secret/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-cert" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368499    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4164bd42-81bf-478d-8409-001aed0783dc" volumeName="kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368509    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9377de09-4111-4499-a1d7-62e407f76422" volumeName="kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-tmp" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368084    5039 factory.go:103] Registering Raw factory
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368522    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9377de09-4111-4499-a1d7-62e407f76422" volumeName="kubernetes.io/projected/9377de09-4111-4499-a1d7-62e407f76422-kube-api-access-hp9v2" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368535    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd5494d4-4343-4f18-b981-b39fe6a54034" volumeName="kubernetes.io/configmap/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-cabundle" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368544    5039 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368546    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="be3ebfff-ac2e-4429-ac1e-1fb87d7599d0" volumeName="kubernetes.io/projected/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-kube-api-access-j7rst" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368581    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcb074c6-4262-4601-9730-f4ae23ee938b" volumeName="kubernetes.io/projected/fcb074c6-4262-4601-9730-f4ae23ee938b-kube-api-access-5j4hz" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368593    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368605    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368618    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec90134-2d9f-418f-98e5-877305e1a0b2" volumeName="kubernetes.io/configmap/8ec90134-2d9f-418f-98e5-877305e1a0b2-telemetry-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368630    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="928e6807-f385-42a2-89d5-e9ed3b1089cf" volumeName="kubernetes.io/configmap/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368644    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" volumeName="kubernetes.io/projected/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-kube-api-access-f8dml" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368663    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5017035-43de-4aec-bcb7-cbdee8be6bc3" volumeName="kubernetes.io/projected/d5017035-43de-4aec-bcb7-cbdee8be6bc3-kube-api-access-692kd" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368674    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df3c83fd-42b4-423d-a2a2-9da642a21118" volumeName="kubernetes.io/projected/df3c83fd-42b4-423d-a2a2-9da642a21118-kube-api-access-nr76c" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368685    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e" volumeName="kubernetes.io/configmap/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-nginx-conf" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368696    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4164bd42-81bf-478d-8409-001aed0783dc" volumeName="kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368706    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4164bd42-81bf-478d-8409-001aed0783dc" volumeName="kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368717    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b546556a-5219-406b-88d0-a3b490c7b259" volumeName="kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-trusted-ca" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368729    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd5494d4-4343-4f18-b981-b39fe6a54034" volumeName="kubernetes.io/secret/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-key" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368739    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7411ec4-7d75-44ad-ae7c-fae47eee36f4" volumeName="kubernetes.io/projected/c7411ec4-7d75-44ad-ae7c-fae47eee36f4-kube-api-access-qp9mj" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368750    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9377de09-4111-4499-a1d7-62e407f76422" volumeName="kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-service-ca-bundle" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368761    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" volumeName="kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368772    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e13536d8-6703-46a7-9e8e-8136e1ff8979" volumeName="kubernetes.io/projected/e13536d8-6703-46a7-9e8e-8136e1ff8979-kube-api-access-fr9hr" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368782    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcb074c6-4262-4601-9730-f4ae23ee938b" volumeName="kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368792    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcb074c6-4262-4601-9730-f4ae23ee938b" volumeName="kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-config" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368803    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="038b383d-62ae-45da-a1a5-dc2246a5d946" volumeName="kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368815    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69b134d1-aa78-45f4-a572-7bc4171c5a56" volumeName="kubernetes.io/projected/69b134d1-aa78-45f4-a572-7bc4171c5a56-kube-api-access-zcsww" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368827    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81cc5d87-5a2f-43e5-ba84-04993c15476e" volumeName="kubernetes.io/projected/81cc5d87-5a2f-43e5-ba84-04993c15476e-kube-api-access-2qvxs" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368838    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec90134-2d9f-418f-98e5-877305e1a0b2" volumeName="kubernetes.io/projected/8ec90134-2d9f-418f-98e5-877305e1a0b2-kube-api-access-q9km5" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368849    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c04efb2b-37d6-4ded-a359-ccb48c497b43" volumeName="kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-tmp" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368860    5039 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c65a7a37-b3e1-46a1-9ed7-204de523fdc7" volumeName="kubernetes.io/projected/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kube-api-access-9sngl" seLinuxMountContext=""
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368872    5039 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368879    5039 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:34:29.369713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.368907    5039 manager.go:319] Starting recovery of all containers
Nov 04 01:34:29.379974 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.379947    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/1.log"
Nov 04 01:34:29.380079 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.380060    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/1.log"
Nov 04 01:34:29.397179 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.397132    5039 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:34:29.398546 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.398525    5039 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:34:29.398546 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.398547    5039 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:34:29.398739 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.398590    5039 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:34:29.398739 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.398631    5039 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 01:34:29.421033 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.420986    5039 generic.go:356] "Generic (PLEG): container finished" podID="9a45018bbca93ea1080adf030d8fbb7f" containerID="3524f371f99d71361dae2e005d3da2297a709c26655dc3970b1351b353d090be" exitCode=0
Nov 04 01:34:29.429749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.429725    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/1.log"
Nov 04 01:34:29.429891 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.429756    5039 generic.go:356] "Generic (PLEG): container finished" podID="b546556a-5219-406b-88d0-a3b490c7b259" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0" exitCode=255
Nov 04 01:34:29.442071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.442032    5039 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="deddbd24546f687ca9eb366a921520e5f02bcf6beb6be7d591802599355a920f" exitCode=0
Nov 04 01:34:29.442071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.442068    5039 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="5a8e1765847b8d10eb3e1926e6965bbdd3c2eac7d798a8d8b998bdfba0107497" exitCode=0
Nov 04 01:34:29.442265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.442082    5039 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="e53cf97be7611f1908941a1102b4a3380e398f29c2d03716689b00fc4fbfaf2d" exitCode=0
Nov 04 01:34:29.442265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.442092    5039 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="d964877aae60435648b96594342e62f9ec630a12e36d8809cc7f514570d9479c" exitCode=0
Nov 04 01:34:29.442265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.442104    5039 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="091b3e2f38d2b9fb179cc887af7fa4027cef193102a14b109a81e0594e1e520b" exitCode=0
Nov 04 01:34:29.442265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.442113    5039 generic.go:356] "Generic (PLEG): container finished" podID="cfbd09a6-31fc-4e11-bf84-da0864ec84ff" containerID="04bebed6bb7f38f5400f0da91809b3c4fbf6f5f6569e20d95ed9309cbc35b314" exitCode=0
Nov 04 01:34:29.465377 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.465350    5039 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:34:29.482062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.481995    5039 manager.go:324] Recovery completed
Nov 04 01:34:29.499397 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.499356    5039 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 01:34:29.501713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.501699    5039 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:29.503513 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.503489    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:29.503674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.503528    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:29.503674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.503543    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:29.506424 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506400    5039 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:34:29.506424 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506419    5039 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:34:29.506634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506438    5039 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:34:29.506634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506595    5039 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 01:34:29.506634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506608    5039 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 01:34:29.506634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506630    5039 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 01:34:29.506800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506638    5039 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 01:34:29.506800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506645    5039 policy_none.go:50] "Start"
Nov 04 01:34:29.506800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506655    5039 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:34:29.506800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506667    5039 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:34:29.506963 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506832    5039 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 01:34:29.506963 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506840    5039 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 01:34:29.506963 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506848    5039 policy_none.go:44] "Start"
Nov 04 01:34:29.507080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.506988    5039 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:29.516828 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.510260    5039 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:29.517165 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.517127    5039 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:34:29.517381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.517359    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:29.517454 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.517385    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:29.517454 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.517400    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:29.522431 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.522419    5039 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:34:29.522627 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.522614    5039 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:34:29.522716 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.522708    5039 server.go:87] "Starting device plugin registration server"
Nov 04 01:34:29.523086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.523072    5039 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:34:29.523195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.523169    5039 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:34:29.525517 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.525499    5039 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:34:29.527052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.527005    5039 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:34:29.527052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.527050    5039 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:34:29.527052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.527056    5039 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:34:29.529807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.527124    5039 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:34:29.527040762Z","UUID":"7f46db9e-1d2d-4fe4-be51-4347e09db31d","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:34:29.529807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.529493    5039 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:34:29.529807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.529524    5039 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:34:29.531297 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.530801    5039 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:34:29.531297 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.530841    5039 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Nov 04 01:34:29.559032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.559001    5039 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:29.559192 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.559071    5039 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:29.559192 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.559167    5039 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 01:34:29.609247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.609224    5039 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:29.624168 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.624141    5039 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:29.625206 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.625187    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:29.625308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.625226    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:29.625308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.625240    5039 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:29.625308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.625296    5039 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.699646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.699591    5039 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"]
Nov 04 01:34:29.700263 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700213    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"541e7628a44bd14f5c6718b103113b2ad3ea2bc3e946b1deab2667ec402dbbee"}
Nov 04 01:34:29.700366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700275    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerDied","Data":"3524f371f99d71361dae2e005d3da2297a709c26655dc3970b1351b353d090be"}
Nov 04 01:34:29.700366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700287    5039 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.700366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700318    5039 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.700366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700293    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"4d58a837f9ca92ab9f681a418a1fd2c2a205b90d719a2d80ab76f51a27df2ca8"}
Nov 04 01:34:29.700597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700373    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"547e2cede1fc8a4092d19734f469e16a00b49ff92f8a8f0d3ba0470812d2c602"}
Nov 04 01:34:29.700597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.700386    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"1e9a3fec3a062b8877d5627c36eff82d4459ef4bb487336cdb5b2034d5b42a13"}
Nov 04 01:34:29.733791 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.733734    5039 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.733889 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.733808    5039 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.762106 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.762081    5039 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:34:29.762240 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.762125    5039 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.762807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.762794    5039 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:34:29.762858 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:29.762828    5039 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.861868 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.861836    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.862072 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.861910    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.862072 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.861954    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.962805 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.962770    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.962805 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.962780    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.962982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.962844    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.962982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.962871    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.962982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.962920    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:29.962982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:29.962955    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:30.345901 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.345872    5039 apiserver.go:52] "Watching apiserver"
Nov 04 01:34:30.450255 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.450215    5039 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:34:30.451033 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.451006    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-sdcdd","openshift-ingress/router-default-54848578cf-gq6dc","openshift-console-operator/console-operator-7cd56b8dbc-wsc5z","openshift-image-registry/image-registry-557fcb4957-k6rbt","openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs","kube-system/global-pull-secret-syncer-p7jkf","openshift-ingress-canary/ingress-canary-m66qn","openshift-insights/periodic-gathering-ttvxq-rrlpl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","kube-system/kas-connection-checker-685895c7c9-j4l4l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7","openshift-cluster-node-tuning-operator/tuned-459wv","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx","openshift-network-diagnostics/network-check-target-68r2l","kube-system/kas-connection-checker-685895c7c9-f66h9","openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8","openshift-image-registry/image-registry-d844f87d4-6hn9v","openshift-multus/multus-8lvcd","openshift-service-ca/service-ca-5fffc97bdf-tlb6z","kube-system/kas-connection-checker-685895c7c9-sx466","openshift-multus/multus-additional-cni-plugins-vsknk","openshift-multus/network-metrics-daemon-2w6dn","openshift-ovn-kubernetes/ovnkube-node-zgrx2","openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq","openshift-dns/node-resolver-tzccz","openshift-network-operator/iptables-alerter-h8x5d","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4","kube-system/konnectivity-agent-56trq","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal","openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn","openshift-insights/insights-runtime-extractor-6xjwj","openshift-dns/dns-default-jn9bb","openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c","openshift-image-registry/node-ca-kl2jb","openshift-insights/insights-operator-7c4986d449-8zdbz"]
Nov 04 01:34:30.452900 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.452879    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.453650 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.453632    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.454528 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.454512    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.458582 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.458546    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.458667 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.458606    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:30.461882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.461866    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.463748 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.463732    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:30.465089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.465070    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.466044 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.466027    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.489234 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.489217    5039 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:30.489355 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.489343    5039 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:30.505511 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505490    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 01:34:30.505632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505491    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.505632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505517    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.505632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505516    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.505632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505545    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 01:34:30.505632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505530    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.505632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505586    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.505897 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.505676    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.506064 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506044    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.506064 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506052    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506071    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506096    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506122    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506124    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506135    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506144    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506168    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506175    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 01:34:30.506218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506170    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 01:34:30.506534 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506337    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 01:34:30.506642 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.506627    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 01:34:30.507179 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.507159    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 01:34:30.557770 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.557737    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.557978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.557964    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:34:30.558056 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.557974    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558056 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.557974    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 01:34:30.558056 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558004    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.558056 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558028    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558091    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558096    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558095    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558096    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558120    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558123    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558053    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558156    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558161    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558171    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558185    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558188    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558198    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558204    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558218    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558223    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558233    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558137    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558249    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.558251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558263    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558275    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558285    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558277    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558289    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558290    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558317    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558323    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558317    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558324    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558367    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558388    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558447    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558511    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558556    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558707    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558790    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 01:34:30.559029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.558854    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:34:30.559555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.559059    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:34:30.559555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.559090    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:34:30.559555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.559148    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 01:34:30.564374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564353    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67jjd\" (UniqueName: \"kubernetes.io/projected/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db-kube-api-access-67jjd\") pod \"kas-connection-checker-685895c7c9-sx466\" (UID: \"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db\") " pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:30.564452 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564382    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.564452 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564401    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.564452 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564419    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp9v2\" (UniqueName: \"kubernetes.io/projected/9377de09-4111-4499-a1d7-62e407f76422-kube-api-access-hp9v2\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.564599 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564467    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-slash\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.564599 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564546    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-run\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.564671 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564595    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.564671 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564650    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/016762ac-ad73-46b0-8059-9b4b2b670043-konnectivity-ca\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:30.564740 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564728    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-trusted-ca\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.564769 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564755    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp9mj\" (UniqueName: \"kubernetes.io/projected/c7411ec4-7d75-44ad-ae7c-fae47eee36f4-kube-api-access-qp9mj\") pod \"volume-data-source-validator-58c9f5f46d-4w7fx\" (UID: \"c7411ec4-7d75-44ad-ae7c-fae47eee36f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:30.564812 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564793    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.564929 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564916    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-config\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.564978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564939    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.564978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564955    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.565068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564977    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:30.565068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.564998    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.565068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565019    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-kubelet\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.565068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565062    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-systemd-units\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.565233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565058    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/016762ac-ad73-46b0-8059-9b4b2b670043-konnectivity-ca\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:30.565233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565093    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsww\" (UniqueName: \"kubernetes.io/projected/69b134d1-aa78-45f4-a572-7bc4171c5a56-kube-api-access-zcsww\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.565233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565125    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j4hz\" (UniqueName: \"kubernetes.io/projected/fcb074c6-4262-4601-9730-f4ae23ee938b-kube-api-access-5j4hz\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.565233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565145    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmsgm\" (UniqueName: \"kubernetes.io/projected/c04efb2b-37d6-4ded-a359-ccb48c497b43-kube-api-access-mmsgm\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.565233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565169    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9jp\" (UniqueName: \"kubernetes.io/projected/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-kube-api-access-wl9jp\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:30.565233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565217    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-socket-dir-parent\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.565478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565273    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.565478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565275    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-config\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.565478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565274    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-dbus\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.565478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565371    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr9hr\" (UniqueName: \"kubernetes.io/projected/e13536d8-6703-46a7-9e8e-8136e1ff8979-kube-api-access-fr9hr\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.565478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565447    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-service-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.565727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565477    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.565727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565539    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-systemd\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.565727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565594    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-bin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.565727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565631    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-daemon-config\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.565727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565656    5039 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:34:30.565937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565733    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-netd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.565937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565758    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.565937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565803    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.565937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565835    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-netns\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.565937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565881    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-daemon-config\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565937    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565962    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-script-lib\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565982    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-kubernetes\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.565998    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566012    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b546556a-5219-406b-88d0-a3b490c7b259-serving-cert\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566044    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87w7x\" (UniqueName: \"kubernetes.io/projected/b546556a-5219-406b-88d0-a3b490c7b259-kube-api-access-87w7x\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566078    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-netns\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.566163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566077    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-service-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566184    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-bin\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566227    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-env-overrides\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566243    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-ovnkube-script-lib\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566255    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566278    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w947m\" (UniqueName: \"kubernetes.io/projected/9b16b031-58ec-4d17-82b1-61a7c952c887-kube-api-access-w947m\") pod \"network-check-source-6f78d59f99-8hlvs\" (UID: \"9b16b031-58ec-4d17-82b1-61a7c952c887\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566305    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6817a7e5-506a-4ae2-8128-5b6601281286-iptables-alerter-script\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566330    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-device-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566353    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-tmp\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566367    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcb074c6-4262-4601-9730-f4ae23ee938b-env-overrides\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566377    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-default-certificate\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566407    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-snapshots\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566428    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566451    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566456    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-tmp\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566471    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df3c83fd-42b4-423d-a2a2-9da642a21118-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:30.566499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566504    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/016762ac-ad73-46b0-8059-9b4b2b670043-agent-certs\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566534    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-config\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566545    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6817a7e5-506a-4ae2-8128-5b6601281286-iptables-alerter-script\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566554    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e13536d8-6703-46a7-9e8e-8136e1ff8979-hosts-file\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566602    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pzgb\" (UniqueName: \"kubernetes.io/projected/07ebf665-177c-48e7-969c-28057a4ff1a4-kube-api-access-4pzgb\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566612    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-default-certificate\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566638    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-snapshots\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566669    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47p5l\" (UniqueName: \"kubernetes.io/projected/928e6807-f385-42a2-89d5-e9ed3b1089cf-kube-api-access-47p5l\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566677    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df3c83fd-42b4-423d-a2a2-9da642a21118-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566751    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rst\" (UniqueName: \"kubernetes.io/projected/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-kube-api-access-j7rst\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566801    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-config\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566839    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-k8s-cni-cncf-io\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566917    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-hostroot\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566939    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-modprobe-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.566965    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-sys\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567011    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvxs\" (UniqueName: \"kubernetes.io/projected/81cc5d87-5a2f-43e5-ba84-04993c15476e-kube-api-access-2qvxs\") pod \"kas-connection-checker-685895c7c9-j4l4l\" (UID: \"81cc5d87-5a2f-43e5-ba84-04993c15476e\") " pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567049    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9km5\" (UniqueName: \"kubernetes.io/projected/8ec90134-2d9f-418f-98e5-877305e1a0b2-kube-api-access-q9km5\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567086    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-multus\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567101    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-multus-certs\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.567136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567122    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-etc-kubernetes\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567199    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567246    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567332    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-var-lib-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567388    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-ovn\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567410    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567433    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567453    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-kubelet\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567467    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysconfig\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567481    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-conf\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567495    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-lib-modules\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567508    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-host\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567523    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-tuned\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567593    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-tmp\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567628    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-tuned\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567638    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-registration-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567665    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567728    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-socket-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567737    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9377de09-4111-4499-a1d7-62e407f76422-tmp\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567762    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-log-socket\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.567781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567781    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-kubelet-config\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567804    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sngl\" (UniqueName: \"kubernetes.io/projected/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kube-api-access-9sngl\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567834    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-cnibin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567901    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567929    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567965    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-node-log\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567983    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/081f052e-3e74-4fd5-8d12-d91aa017a999-config\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.567997    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-sys-fs\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568011    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-systemd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568027    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568048    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-host\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568071    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-serviceca\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568120    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568141    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6zm\" (UniqueName: \"kubernetes.io/projected/6817a7e5-506a-4ae2-8128-5b6601281286-kube-api-access-zq6zm\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568180    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d5f\" (UniqueName: \"kubernetes.io/projected/6a1464fb-a344-4bd9-8406-6fe464ff4fa0-kube-api-access-l4d5f\") pod \"kas-connection-checker-685895c7c9-f66h9\" (UID: \"6a1464fb-a344-4bd9-8406-6fe464ff4fa0\") " pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568197    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df3c83fd-42b4-423d-a2a2-9da642a21118-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568204    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/081f052e-3e74-4fd5-8d12-d91aa017a999-config\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568218    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568269    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568270    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568285    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-serviceca\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568312    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568407    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcb074c6-4262-4601-9730-f4ae23ee938b-ovn-node-metrics-cert\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568446    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df3c83fd-42b4-423d-a2a2-9da642a21118-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568452    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.568475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568487    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8ec90134-2d9f-418f-98e5-877305e1a0b2-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568510    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568528    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-conf-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568556    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-var-lib-kubelet\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568615    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/081f052e-3e74-4fd5-8d12-d91aa017a999-serving-cert\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568650    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568719    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e13536d8-6703-46a7-9e8e-8136e1ff8979-tmp-dir\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568714    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8ec90134-2d9f-418f-98e5-877305e1a0b2-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568751    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9377de09-4111-4499-a1d7-62e407f76422-serving-cert\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568762    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e13536d8-6703-46a7-9e8e-8136e1ff8979-tmp-dir\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568772    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-system-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568809    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568838    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568872    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568864    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568914    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-cni-binary-copy\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568926    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568941    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.568978    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mc9h\" (UniqueName: \"kubernetes.io/projected/081f052e-3e74-4fd5-8d12-d91aa017a999-kube-api-access-6mc9h\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569010    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569037    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569072    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69b134d1-aa78-45f4-a572-7bc4171c5a56-cni-binary-copy\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569082    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8dml\" (UniqueName: \"kubernetes.io/projected/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-kube-api-access-f8dml\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569156    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9377de09-4111-4499-a1d7-62e407f76422-serving-cert\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569154    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-nginx-conf\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569179    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9377de09-4111-4499-a1d7-62e407f76422-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569182    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569214    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-system-cni-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569230    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cnibin\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569272    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569291    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-etc-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569306    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569311    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-nginx-conf\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569347    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr76c\" (UniqueName: \"kubernetes.io/projected/df3c83fd-42b4-423d-a2a2-9da642a21118-kube-api-access-nr76c\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569379    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-stats-auth\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569406    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6817a7e5-506a-4ae2-8128-5b6601281286-host-slash\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.569411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569419    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.570461 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.569667    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-stats-auth\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.607343 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607264    5039 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:34:30.607343 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607277    5039 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:34:30.607493 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607292    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 01:34:30.607526 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607503    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9rdvr\""
Nov 04 01:34:30.607526 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607367    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 01:34:30.607639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607388    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:34:30.607639 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.607420    5039 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:30.607639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607596    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:34:30.607639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607601    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 01:34:30.607820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607672    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 01:34:30.607820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607681    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 01:34:30.607820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607710    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 01:34:30.607820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607716    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:34:30.607820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607722    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ctj2f\""
Nov 04 01:34:30.607820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607779    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-sj5vk\""
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.607326    5039 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607902    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607845    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607951    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vqsf7\""
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.607966    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608006    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:34:30.608077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608060    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608103    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608114    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5jndn\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608119    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jhtpj\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608135    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608136    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mh5l6\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608144    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vr5s4\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608174    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ldpmc\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608185    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lhd4j\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608178    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xwn9r\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608216    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r7ngl\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608226    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tfqdd\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608237    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xkhwh\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608247    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608227    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rx6cn\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608247    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608269    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-shfxh\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608276    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gnb5c\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608310    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-4rqhb\""
Nov 04 01:34:30.608352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608354    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zbbgd\""
Nov 04 01:34:30.608841 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608383    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j6ln8\""
Nov 04 01:34:30.608951 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.608937    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.609065 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.609050    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcb074c6-4262-4601-9730-f4ae23ee938b-ovn-node-metrics-cert\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.609104 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.609086    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.609218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.609202    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.609257 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.609241    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/081f052e-3e74-4fd5-8d12-d91aa017a999-serving-cert\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:30.609343 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.609331    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.615613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.615596    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b546556a-5219-406b-88d0-a3b490c7b259-trusted-ca\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.615697 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.615651    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76b11319-689c-4ba5-83ec-afe254b80174-original-pull-secret\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.616680 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.616661    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b546556a-5219-406b-88d0-a3b490c7b259-serving-cert\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.616863 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.616846    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/016762ac-ad73-46b0-8059-9b4b2b670043-agent-certs\") pod \"konnectivity-agent-56trq\" (UID: \"016762ac-ad73-46b0-8059-9b4b2b670043\") " pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:30.616912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.616886    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.617479 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.617465    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.617650 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.617635    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m66qn_d4f41073-2c64-44f1-9153-ca4e0e2c0b50/serve-healthcheck-canary/0.log"
Nov 04 01:34:30.622428 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.622413    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp9v2\" (UniqueName: \"kubernetes.io/projected/9377de09-4111-4499-a1d7-62e407f76422-kube-api-access-hp9v2\") pod \"insights-operator-7c4986d449-8zdbz\" (UID: \"9377de09-4111-4499-a1d7-62e407f76422\") " pod="openshift-insights/insights-operator-7c4986d449-8zdbz"
Nov 04 01:34:30.623282 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.623267    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67jjd\" (UniqueName: \"kubernetes.io/projected/fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db-kube-api-access-67jjd\") pod \"kas-connection-checker-685895c7c9-sx466\" (UID: \"fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db\") " pod="kube-system/kas-connection-checker-685895c7c9-sx466"
Nov 04 01:34:30.623925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.623909    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9jp\" (UniqueName: \"kubernetes.io/projected/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-kube-api-access-wl9jp\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:30.625258 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.625234    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmsgm\" (UniqueName: \"kubernetes.io/projected/c04efb2b-37d6-4ded-a359-ccb48c497b43-kube-api-access-mmsgm\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.625396 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.625309    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.625396 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.625375    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcsww\" (UniqueName: \"kubernetes.io/projected/69b134d1-aa78-45f4-a572-7bc4171c5a56-kube-api-access-zcsww\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.625470 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.625454    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr9hr\" (UniqueName: \"kubernetes.io/projected/e13536d8-6703-46a7-9e8e-8136e1ff8979-kube-api-access-fr9hr\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.625514 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.625500    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87w7x\" (UniqueName: \"kubernetes.io/projected/b546556a-5219-406b-88d0-a3b490c7b259-kube-api-access-87w7x\") pod \"console-operator-7cd56b8dbc-wsc5z\" (UID: \"b546556a-5219-406b-88d0-a3b490c7b259\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:30.625635 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.625618    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp9mj\" (UniqueName: \"kubernetes.io/projected/c7411ec4-7d75-44ad-ae7c-fae47eee36f4-kube-api-access-qp9mj\") pod \"volume-data-source-validator-58c9f5f46d-4w7fx\" (UID: \"c7411ec4-7d75-44ad-ae7c-fae47eee36f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4w7fx"
Nov 04 01:34:30.627649 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.627631    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j4hz\" (UniqueName: \"kubernetes.io/projected/fcb074c6-4262-4601-9730-f4ae23ee938b-kube-api-access-5j4hz\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.631380 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.631347    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rst\" (UniqueName: \"kubernetes.io/projected/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-kube-api-access-j7rst\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.631480 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.631459    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w947m\" (UniqueName: \"kubernetes.io/projected/9b16b031-58ec-4d17-82b1-61a7c952c887-kube-api-access-w947m\") pod \"network-check-source-6f78d59f99-8hlvs\" (UID: \"9b16b031-58ec-4d17-82b1-61a7c952c887\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-8hlvs"
Nov 04 01:34:30.631516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.631483    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47p5l\" (UniqueName: \"kubernetes.io/projected/928e6807-f385-42a2-89d5-e9ed3b1089cf-kube-api-access-47p5l\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.631613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.631599    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pzgb\" (UniqueName: \"kubernetes.io/projected/07ebf665-177c-48e7-969c-28057a4ff1a4-kube-api-access-4pzgb\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.631715 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.631692    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.664332 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.664308    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvxs\" (UniqueName: \"kubernetes.io/projected/81cc5d87-5a2f-43e5-ba84-04993c15476e-kube-api-access-2qvxs\") pod \"kas-connection-checker-685895c7c9-j4l4l\" (UID: \"81cc5d87-5a2f-43e5-ba84-04993c15476e\") " pod="kube-system/kas-connection-checker-685895c7c9-j4l4l"
Nov 04 01:34:30.664721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.664683    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6zm\" (UniqueName: \"kubernetes.io/projected/6817a7e5-506a-4ae2-8128-5b6601281286-kube-api-access-zq6zm\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.664873 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.664852    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9km5\" (UniqueName: \"kubernetes.io/projected/8ec90134-2d9f-418f-98e5-877305e1a0b2-kube-api-access-q9km5\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.665117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665096    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mc9h\" (UniqueName: \"kubernetes.io/projected/081f052e-3e74-4fd5-8d12-d91aa017a999-kube-api-access-6mc9h\") pod \"service-ca-operator-544d749c8c-k2l7c\" (UID: \"081f052e-3e74-4fd5-8d12-d91aa017a999\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c"
Nov 04 01:34:30.665224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665163    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8dml\" (UniqueName: \"kubernetes.io/projected/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-kube-api-access-f8dml\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.665316 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665246    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4d5f\" (UniqueName: \"kubernetes.io/projected/6a1464fb-a344-4bd9-8406-6fe464ff4fa0-kube-api-access-l4d5f\") pod \"kas-connection-checker-685895c7c9-f66h9\" (UID: \"6a1464fb-a344-4bd9-8406-6fe464ff4fa0\") " pod="kube-system/kas-connection-checker-685895c7c9-f66h9"
Nov 04 01:34:30.665316 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665287    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sngl\" (UniqueName: \"kubernetes.io/projected/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kube-api-access-9sngl\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.665438 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665397    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr76c\" (UniqueName: \"kubernetes.io/projected/df3c83fd-42b4-423d-a2a2-9da642a21118-kube-api-access-nr76c\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4\" (UID: \"df3c83fd-42b4-423d-a2a2-9da642a21118\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4"
Nov 04 01:34:30.665438 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665408    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq5xp\" (UniqueName: \"kubernetes.io/projected/a5bea9b9-e510-4b8b-bc95-5ad38581b40c-kube-api-access-fq5xp\") pod \"network-check-target-68r2l\" (UID: \"a5bea9b9-e510-4b8b-bc95-5ad38581b40c\") " pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:30.665554 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665513    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.665797 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.665778    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.670623 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670604    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-modprobe-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.670707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670629    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-sys\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.670707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670646    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-multus\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.670707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670659    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-multus-certs\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.670707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670698    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-multus-certs\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.670707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670702    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-multus\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670711    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-sys\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670740    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-etc-kubernetes\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670765    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-var-lib-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670780    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-ovn\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670790    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-etc-kubernetes\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670794    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670796    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-modprobe-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670805    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-var-lib-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670814    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-ovn\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670815    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-kubelet\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670840    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysconfig\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670842    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-kubelet\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670871    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-conf\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.670894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670887    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysconfig\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670904    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-lib-modules\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670925    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-host\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.670904    5039 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670981    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-conf\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670991    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-registration-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671000    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-host\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670999    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-lib-modules\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.670945    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-registration-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.671030    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.171014792 +0000 UTC m=+1.887201652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671089    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671118    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tld9\" (UniqueName: \"kubernetes.io/projected/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea-kube-api-access-4tld9\") pod \"migrator-66df5d496b-42wch\" (UID: \"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671141    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671161    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-socket-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671178    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-log-socket\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671193    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-kubelet-config\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671211    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-cnibin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671228    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671235    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671245    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-socket-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671253    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-log-socket\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671272    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671296    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-cnibin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671298    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-node-log\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671329    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-node-log\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671334    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw9md\" (UniqueName: \"kubernetes.io/projected/bd5494d4-4343-4f18-b981-b39fe6a54034-kube-api-access-mw9md\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671300    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-kubelet-config\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.671301    5039 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:34:30.671347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671362    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671382    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-sysctl-d\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671400    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-sys-fs\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.671431    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.171405543 +0000 UTC m=+1.887592399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : secret "metrics-daemon-secret" not found
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671446    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-sys-fs\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671466    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-systemd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671489    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5z6n\" (UniqueName: \"kubernetes.io/projected/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-kube-api-access-j5z6n\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671506    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-host\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671512    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-systemd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671523    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-cabundle\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671545    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671549    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be3ebfff-ac2e-4429-ac1e-1fb87d7599d0-host\") pod \"node-ca-kl2jb\" (UID: \"be3ebfff-ac2e-4429-ac1e-1fb87d7599d0\") " pod="openshift-image-registry/node-ca-kl2jb"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671559    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671603    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.671628    5039 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671633    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.671664    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.171648865 +0000 UTC m=+1.887835719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671689    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671707    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671715    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671693    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-cabundle\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671761    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-conf-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671738    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-conf-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671774    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671783    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-var-lib-kubelet\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671809    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671812    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-key\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671861    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-var-lib-kubelet\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671861    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-config-volume\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671901    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-system-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671933    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/909efb32-95fc-4b85-bf58-949637c49c16-crio-socket\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.671975    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bd5494d4-4343-4f18-b981-b39fe6a54034-signing-key\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672019    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672319    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-config-volume\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672375    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-system-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.672392    5039 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672397    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-system-cni-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.672407    5039 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672435    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cnibin\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672463    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-etc-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.672490    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.172474592 +0000 UTC m=+1.888661444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672508    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-system-cni-dir\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672514    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-etc-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.672482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672518    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672614    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfbd09a6-31fc-4e11-bf84-da0864ec84ff-cnibin\") pod \"multus-additional-cni-plugins-vsknk\" (UID: \"cfbd09a6-31fc-4e11-bf84-da0864ec84ff\") " pod="openshift-multus/multus-additional-cni-plugins-vsknk"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672624    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672670    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6817a7e5-506a-4ae2-8128-5b6601281286-host-slash\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672708    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672725    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672737    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672761    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-slash\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672788    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-run\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672805    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6817a7e5-506a-4ae2-8128-5b6601281286-host-slash\") pod \"iptables-alerter-h8x5d\" (UID: \"6817a7e5-506a-4ae2-8128-5b6601281286\") " pod="openshift-network-operator/iptables-alerter-h8x5d"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672817    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-692kd\" (UniqueName: \"kubernetes.io/projected/d5017035-43de-4aec-bcb7-cbdee8be6bc3-kube-api-access-692kd\") pod \"migrator-66df5d496b-97prq\" (UID: \"d5017035-43de-4aec-bcb7-cbdee8be6bc3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672876    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672906    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672938    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-kubelet\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.672973    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-systemd-units\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673008    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-tmp-dir\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673040    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-socket-dir-parent\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673066    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-dbus\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673099    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.673135    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.173119323 +0000 UTC m=+1.889306173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673164    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbz6\" (UniqueName: \"kubernetes.io/projected/909efb32-95fc-4b85-bf58-949637c49c16-kube-api-access-jcbz6\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673184    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673202    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673255    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-slash\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.673287    5039 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673313    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-run\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.673390    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.173372629 +0000 UTC m=+1.889559477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673458    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-cni-dir\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673539    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-systemd-units\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673555    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-multus-socket-dir-parent\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673623    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76b11319-689c-4ba5-83ec-afe254b80174-dbus\") pod \"global-pull-secret-syncer-p7jkf\" (UID: \"76b11319-689c-4ba5-83ec-afe254b80174\") " pod="kube-system/global-pull-secret-syncer-p7jkf"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673650    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-tmp-dir\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673721    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-run-openvswitch\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.673810    5039 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673849    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-systemd\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673875    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-kubelet\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673882    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-bin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673923    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-netd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673931    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-systemd\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.673961    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.173948137 +0000 UTC m=+1.890134972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673985    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-netns\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.673987    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-netd\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674034    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-var-lib-cni-bin\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.674375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674146    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-run-netns\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674419    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674451    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-kubernetes\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674480    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-cert\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674511    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-netns\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674532    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-bin\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674561    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74628\" (UniqueName: \"kubernetes.io/projected/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-kube-api-access-74628\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674617    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-device-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674649    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674680    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674705    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/909efb32-95fc-4b85-bf58-949637c49c16-data-volume\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674735    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e13536d8-6703-46a7-9e8e-8136e1ff8979-hosts-file\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674767    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-k8s-cni-cncf-io\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674793    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-hostroot\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674884    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-hostroot\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.674967    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675024    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c04efb2b-37d6-4ded-a359-ccb48c497b43-etc-kubernetes\") pod \"tuned-459wv\" (UID: \"c04efb2b-37d6-4ded-a359-ccb48c497b43\") " pod="openshift-cluster-node-tuning-operator/tuned-459wv"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675247    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-cert\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.675337    5039 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.675350    5039 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.675407    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.175391785 +0000 UTC m=+1.891578657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675484    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/909efb32-95fc-4b85-bf58-949637c49c16-data-volume\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675536    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcb074c6-4262-4601-9730-f4ae23ee938b-host-cni-bin\") pod \"ovnkube-node-zgrx2\" (UID: \"fcb074c6-4262-4601-9730-f4ae23ee938b\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675617    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e13536d8-6703-46a7-9e8e-8136e1ff8979-hosts-file\") pod \"node-resolver-tzccz\" (UID: \"e13536d8-6703-46a7-9e8e-8136e1ff8979\") " pod="openshift-dns/node-resolver-tzccz"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675669    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-k8s-cni-cncf-io\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675484    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69b134d1-aa78-45f4-a572-7bc4171c5a56-host-run-netns\") pod \"multus-8lvcd\" (UID: \"69b134d1-aa78-45f4-a572-7bc4171c5a56\") " pod="openshift-multus/multus-8lvcd"
Nov 04 01:34:30.676214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.675755    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c65a7a37-b3e1-46a1-9ed7-204de523fdc7-device-dir\") pod \"aws-ebs-csi-driver-node-w8rr7\" (UID: \"c65a7a37-b3e1-46a1-9ed7-204de523fdc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8rr7"
Nov 04 01:34:30.731766 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.731730    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5z6n\" (UniqueName: \"kubernetes.io/projected/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-kube-api-access-j5z6n\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.731881 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.731730    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tld9\" (UniqueName: \"kubernetes.io/projected/7896c15f-392c-49fb-8d4c-9fdbbc3de6ea-kube-api-access-4tld9\") pod \"migrator-66df5d496b-42wch\" (UID: \"7896c15f-392c-49fb-8d4c-9fdbbc3de6ea\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-42wch"
Nov 04 01:34:30.731881 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.731831    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw9md\" (UniqueName: \"kubernetes.io/projected/bd5494d4-4343-4f18-b981-b39fe6a54034-kube-api-access-mw9md\") pod \"service-ca-5fffc97bdf-tlb6z\" (UID: \"bd5494d4-4343-4f18-b981-b39fe6a54034\") " pod="openshift-service-ca/service-ca-5fffc97bdf-tlb6z"
Nov 04 01:34:30.732445 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.732428    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbz6\" (UniqueName: \"kubernetes.io/projected/909efb32-95fc-4b85-bf58-949637c49c16-kube-api-access-jcbz6\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.733000 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.732979    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-692kd\" (UniqueName: \"kubernetes.io/projected/d5017035-43de-4aec-bcb7-cbdee8be6bc3-kube-api-access-692kd\") pod \"migrator-66df5d496b-97prq\" (UID: \"d5017035-43de-4aec-bcb7-cbdee8be6bc3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-97prq"
Nov 04 01:34:30.733185 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.733167    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") pod \"periodic-gathering-ttvxq-rrlpl\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") " pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:34:30.734289 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.734273    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74628\" (UniqueName: \"kubernetes.io/projected/d4f41073-2c64-44f1-9153-ca4e0e2c0b50-kube-api-access-74628\") pod \"ingress-canary-m66qn\" (UID: \"d4f41073-2c64-44f1-9153-ca4e0e2c0b50\") " pod="openshift-ingress-canary/ingress-canary-m66qn"
Nov 04 01:34:30.756329 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.756307    5039 scope.go:122] "RemoveContainer" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0"
Nov 04 01:34:30.775479 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.775456    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.775594 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.775510    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/909efb32-95fc-4b85-bf58-949637c49c16-crio-socket\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.775672 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.775654    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/909efb32-95fc-4b85-bf58-949637c49c16-crio-socket\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.775707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.775679    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.775747 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.775725    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:30.775781 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.775748    5039 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:30.775825 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.775805    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.275792605 +0000 UTC m=+1.991979442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:30.775872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.775817    5039 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:34:30.775872 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.775826    5039 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:34:30.775939 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:30.775896    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:31.275880669 +0000 UTC m=+1.992067504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : configmap "kube-rbac-proxy" not found
Nov 04 01:34:30.778124 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.778105    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b79cfe-4f1c-4786-9f97-e87ce36cbba8-metrics-tls\") pod \"dns-default-jn9bb\" (UID: \"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8\") " pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:30.819034 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:30.819015    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-42wch_7896c15f-392c-49fb-8d4c-9fdbbc3de6ea/migrator/0.log"
Nov 04 01:34:31.018791 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.018763    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-97prq_d5017035-43de-4aec-bcb7-cbdee8be6bc3/migrator/0.log"
Nov 04 01:34:31.065706 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.065678    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:31.179674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.179602    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:31.179674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.179648    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:31.179839 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179736    5039 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:31.179839 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179736    5039 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:31.179839 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.179766    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:31.179839 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179796    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls podName:8ec90134-2d9f-418f-98e5-877305e1a0b2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.179783062 +0000 UTC m=+2.895969897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-j4lwn" (UID: "8ec90134-2d9f-418f-98e5-877305e1a0b2") : secret "cluster-monitoring-operator-tls" not found
Nov 04 01:34:31.179839 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.179814    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:31.179839 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179746    5039 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.179854    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179870    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.179853489 +0000 UTC m=+2.896040327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179883    5039 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179899    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.17988964 +0000 UTC m=+2.896076474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179909    5039 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179917    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.179908607 +0000 UTC m=+2.896095446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.179956    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.179945308 +0000 UTC m=+2.896132157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.179988    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:31.180032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.180026    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.180058    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180110    5039 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180118    5039 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180129    5039 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180141    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.180133806 +0000 UTC m=+2.896320640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180141    5039 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180164    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.180152519 +0000 UTC m=+2.896339357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:31.180309 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.180180    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.18017008 +0000 UTC m=+2.896356953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : secret "metrics-daemon-secret" not found
Nov 04 01:34:31.218532 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.218503    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4_df3c83fd-42b4-423d-a2a2-9da642a21118/kube-storage-version-migrator-operator/0.log"
Nov 04 01:34:31.280897 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.280861    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:31.281047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.280919    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:31.281047 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.281008    5039 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:31.281047 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.281038    5039 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 01:34:31.281151 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.281062    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.281049251 +0000 UTC m=+2.997236086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:31.281151 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.281082    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:32.281067643 +0000 UTC m=+2.997254496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : configmap "kube-rbac-proxy" not found
Nov 04 01:34:31.323773 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.323728    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:31.434287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.434203    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jn9bb"]
Nov 04 01:34:31.437940 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:31.437912    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b79cfe_4f1c_4786_9f97_e87ce36cbba8.slice/crio-67a7e599e640e352e64deab4b6e4ac79eae38398f288b80b1acfbc03f2ab4998 WatchSource:0}: Error finding container 67a7e599e640e352e64deab4b6e4ac79eae38398f288b80b1acfbc03f2ab4998: Status 404 returned error can't find the container with id 67a7e599e640e352e64deab4b6e4ac79eae38398f288b80b1acfbc03f2ab4998
Nov 04 01:34:31.439917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.439901    5039 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:34:31.494076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.494049    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:34:31.494466 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.494451    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/1.log"
Nov 04 01:34:31.494506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.494482    5039 generic.go:356] "Generic (PLEG): container finished" podID="b546556a-5219-406b-88d0-a3b490c7b259" containerID="cddec82cb38bf22d014e294887ef48d0b99d607c26fa4fdb895362e956eeaf5e" exitCode=255
Nov 04 01:34:31.494587 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.494558    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" event={"ID":"b546556a-5219-406b-88d0-a3b490c7b259","Type":"ContainerDied","Data":"cddec82cb38bf22d014e294887ef48d0b99d607c26fa4fdb895362e956eeaf5e"}
Nov 04 01:34:31.494634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.494622    5039 scope.go:122] "RemoveContainer" containerID="99039c63994520e85b2e3d6e71e453abd22af08b417690e7b3cc58c2eca72eb0"
Nov 04 01:34:31.494936 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.494912    5039 scope.go:122] "RemoveContainer" containerID="cddec82cb38bf22d014e294887ef48d0b99d607c26fa4fdb895362e956eeaf5e"
Nov 04 01:34:31.495153 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:31.495123    5039 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wsc5z_openshift-console-operator(b546556a-5219-406b-88d0-a3b490c7b259)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259"
Nov 04 01:34:31.495779 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.495757    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jn9bb" event={"ID":"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8","Type":"ContainerStarted","Data":"67a7e599e640e352e64deab4b6e4ac79eae38398f288b80b1acfbc03f2ab4998"}
Nov 04 01:34:31.586782 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.586756    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:31.838731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:31.838645    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:32.083087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.083051    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:32.189269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189218    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:32.189434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189298    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:32.189434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189348    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:32.189434 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189369    5039 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:34:32.189434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189382    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:32.189434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189410    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189437    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.189418442 +0000 UTC m=+4.905605289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : secret "metrics-daemon-secret" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189478    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189499    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.18948751 +0000 UTC m=+4.905674345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189525    5039 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189534    5039 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189541    5039 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d844f87d4-6hn9v: secret "image-registry-tls" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189560    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189592    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.189555786 +0000 UTC m=+4.905742634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189616    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls podName:038b383d-62ae-45da-a1a5-dc2246a5d946 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.189605558 +0000 UTC m=+4.905792392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls") pod "image-registry-d844f87d4-6hn9v" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946") : secret "image-registry-tls" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189627    5039 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189643    5039 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189654    5039 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557fcb4957-k6rbt: secret "image-registry-tls" not found
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.189667    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:32.189693 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189682    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls podName:cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.189673042 +0000 UTC m=+4.905859890 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls") pod "image-registry-557fcb4957-k6rbt" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2") : secret "image-registry-tls" not found
Nov 04 01:34:32.190138 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189723    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.189703469 +0000 UTC m=+4.905890319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:32.190138 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189737    5039 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:32.190138 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.189766    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.189758165 +0000 UTC m=+4.905945000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:32.191897 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.191881    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ec90134-2d9f-418f-98e5-877305e1a0b2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-j4lwn\" (UID: \"8ec90134-2d9f-418f-98e5-877305e1a0b2\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:32.255422 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.255399    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"
Nov 04 01:34:32.290118 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.290090    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:32.290249 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.290135    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:32.290305 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.290251    5039 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:32.290352 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:32.290334    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:34.290312581 +0000 UTC m=+5.006499435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:32.290647 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.290630    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/909efb32-95fc-4b85-bf58-949637c49c16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:32.328553 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.328522    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:32.502828 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.502758    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:34:32.628349 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.628316    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn"]
Nov 04 01:34:32.632288 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:32.632256    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec90134_2d9f_418f_98e5_877305e1a0b2.slice/crio-e938ae316435d4168979f68fee6be304af580317e3687caa84446db9d4a2391e WatchSource:0}: Error finding container e938ae316435d4168979f68fee6be304af580317e3687caa84446db9d4a2391e: Status 404 returned error can't find the container with id e938ae316435d4168979f68fee6be304af580317e3687caa84446db9d4a2391e
Nov 04 01:34:32.753837 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:32.753760    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:34:33.129664 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:33.129623    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:33.376488 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:33.376452    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:33.507474 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:33.507440    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn" event={"ID":"8ec90134-2d9f-418f-98e5-877305e1a0b2","Type":"ContainerStarted","Data":"e938ae316435d4168979f68fee6be304af580317e3687caa84446db9d4a2391e"}
Nov 04 01:34:33.623169 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:33.623118    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:33.871896 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:33.871864    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:34.119027 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.118998    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:34.209527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209451    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:34.209527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209494    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:34.209527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209518    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209554    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209624    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209657    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:38.209638118 +0000 UTC m=+8.925824953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209701    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209707    5039 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.209750    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209769    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:38.209751962 +0000 UTC m=+8.925938804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:34.209814 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209807    5039 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 01:34:34.210155 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209707    5039 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 01:34:34.210155 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209845    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs podName:10bc58fa-af70-41c2-9e8c-0cf7774b67fe nodeName:}" failed. No retries permitted until 2026-11-04 01:34:38.209832986 +0000 UTC m=+8.926019821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs") pod "network-metrics-daemon-2w6dn" (UID: "10bc58fa-af70-41c2-9e8c-0cf7774b67fe") : secret "metrics-daemon-secret" not found
Nov 04 01:34:34.210155 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209872    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert podName:f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e nodeName:}" failed. No retries permitted until 2026-11-04 01:34:38.209861602 +0000 UTC m=+8.926048443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-sdcdd" (UID: "f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e") : secret "networking-console-plugin-cert" not found
Nov 04 01:34:34.210155 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209891    5039 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 01:34:34.210155 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.209924    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:38.209914563 +0000 UTC m=+8.926101401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : secret "router-metrics-certs-default" not found
Nov 04 01:34:34.212648 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.212625    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"image-registry-d844f87d4-6hn9v\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") " pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:34.212969 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.212947    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"image-registry-557fcb4957-k6rbt\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") " pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:34.311047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.311010    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:34.311219 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.311159    5039 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:34.311277 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:34.311232    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:38.311213984 +0000 UTC m=+9.027400830 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:34.359323 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.359105    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:34.360416 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.360352    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:34.362312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.362289    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:34.514221 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.514116    5039 generic.go:356] "Generic (PLEG): container finished" podID="081f052e-3e74-4fd5-8d12-d91aa017a999" containerID="2e25156a42e00345b09abf30c20c5576ee5674cbf3fe3badba37b5744a5f8805" exitCode=0
Nov 04 01:34:34.514221 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.514200    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c" event={"ID":"081f052e-3e74-4fd5-8d12-d91aa017a999","Type":"ContainerDied","Data":"2e25156a42e00345b09abf30c20c5576ee5674cbf3fe3badba37b5744a5f8805"}
Nov 04 01:34:34.514942 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.514920    5039 scope.go:122] "RemoveContainer" containerID="2e25156a42e00345b09abf30c20c5576ee5674cbf3fe3badba37b5744a5f8805"
Nov 04 01:34:34.518461 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.518437    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jn9bb" event={"ID":"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8","Type":"ContainerStarted","Data":"a55ceb55de71b47e0b18b99ef63179ad52632fe776da2c376c65a8b95e3f3b43"}
Nov 04 01:34:34.518547 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.518487    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jn9bb" event={"ID":"d3b79cfe-4f1c-4786-9f97-e87ce36cbba8","Type":"ContainerStarted","Data":"a2a8d200f9c0e6849a0e33717eba3eeb2c7f9a207f3f17d0e6f66918e613d9b2"}
Nov 04 01:34:34.518874 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.518811    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:34.520441 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.520423    5039 generic.go:356] "Generic (PLEG): container finished" podID="9377de09-4111-4499-a1d7-62e407f76422" containerID="f0615d2882a76e671de900b632848e274e81bd120813919af48abaed3d11d57d" exitCode=0
Nov 04 01:34:34.520502 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.520488    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8zdbz" event={"ID":"9377de09-4111-4499-a1d7-62e407f76422","Type":"ContainerDied","Data":"f0615d2882a76e671de900b632848e274e81bd120813919af48abaed3d11d57d"}
Nov 04 01:34:34.520787 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.520774    5039 scope.go:122] "RemoveContainer" containerID="f0615d2882a76e671de900b632848e274e81bd120813919af48abaed3d11d57d"
Nov 04 01:34:34.815604 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.815514    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d844f87d4-6hn9v"]
Nov 04 01:34:34.819639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.819602    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557fcb4957-k6rbt"]
Nov 04 01:34:34.820370 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:34.820340    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038b383d_62ae_45da_a1a5_dc2246a5d946.slice/crio-cb3d25ea3435d6c2922d2cb027cd9eddb84c01846adf116859911fc73a244450 WatchSource:0}: Error finding container cb3d25ea3435d6c2922d2cb027cd9eddb84c01846adf116859911fc73a244450: Status 404 returned error can't find the container with id cb3d25ea3435d6c2922d2cb027cd9eddb84c01846adf116859911fc73a244450
Nov 04 01:34:34.854651 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:34.854623    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1fd32d_06a3_4b5c_8c3c_dc6b3580b8e2.slice/crio-f4cd2e40b338bbf95de986b22de2ccb11c5f6127fde9bfc3a4d51e2fec5cdb3e WatchSource:0}: Error finding container f4cd2e40b338bbf95de986b22de2ccb11c5f6127fde9bfc3a4d51e2fec5cdb3e: Status 404 returned error can't find the container with id f4cd2e40b338bbf95de986b22de2ccb11c5f6127fde9bfc3a4d51e2fec5cdb3e
Nov 04 01:34:34.900548 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:34.900513    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:35.317730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.317704    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:35.353316 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.353289    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:35.526460 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.526430    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-k2l7c" event={"ID":"081f052e-3e74-4fd5-8d12-d91aa017a999","Type":"ContainerStarted","Data":"c9c747a8c19c6b75117ff01ff3ae66e8b425b2616449c3669888b9f20d03d15b"}
Nov 04 01:34:35.528143 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.528117    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt" event={"ID":"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2","Type":"ContainerStarted","Data":"1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d"}
Nov 04 01:34:35.528268 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.528160    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt" event={"ID":"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2","Type":"ContainerStarted","Data":"f4cd2e40b338bbf95de986b22de2ccb11c5f6127fde9bfc3a4d51e2fec5cdb3e"}
Nov 04 01:34:35.529631 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.529607    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v" event={"ID":"038b383d-62ae-45da-a1a5-dc2246a5d946","Type":"ContainerStarted","Data":"fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071"}
Nov 04 01:34:35.529726 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.529642    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v" event={"ID":"038b383d-62ae-45da-a1a5-dc2246a5d946","Type":"ContainerStarted","Data":"cb3d25ea3435d6c2922d2cb027cd9eddb84c01846adf116859911fc73a244450"}
Nov 04 01:34:35.531426 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.531394    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8zdbz" event={"ID":"9377de09-4111-4499-a1d7-62e407f76422","Type":"ContainerStarted","Data":"83b62dcb0bc614eb501283fb62186b9aabb1e8f2cce08d37daec27dbb3efcf65"}
Nov 04 01:34:35.533060 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.533037    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-j4lwn" event={"ID":"8ec90134-2d9f-418f-98e5-877305e1a0b2","Type":"ContainerStarted","Data":"f73907ea42ea18796490f8fe916b0cdf6cf547c7c7fac078fb8899c613abea69"}
Nov 04 01:34:35.603995 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.603917    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:35.674752 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.674716    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:35.846576 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.846540    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:35.969794 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:35.969722    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:36.091738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:36.091708    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:36.344866 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:36.344835    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:36.595633 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:36.595549    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:36.843735 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:36.843699    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:36.993299 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:36.993219    5039 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:36.993699 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:36.993681    5039 scope.go:122] "RemoveContainer" containerID="cddec82cb38bf22d014e294887ef48d0b99d607c26fa4fdb895362e956eeaf5e"
Nov 04 01:34:36.993925 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:36.993903    5039 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wsc5z_openshift-console-operator(b546556a-5219-406b-88d0-a3b490c7b259)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259"
Nov 04 01:34:37.354670 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:37.354632    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:37.602165 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:37.602126    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:37.854280 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:37.854204    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:38.099815 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.099785    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:38.253504 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.253415    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:38.253504 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.253475    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:38.253504 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.253497    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:38.253798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.253520    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:38.253798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.253559    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:38.253798 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:38.253647    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle podName:07ebf665-177c-48e7-969c-28057a4ff1a4 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:46.253624106 +0000 UTC m=+16.969810955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle") pod "router-default-54848578cf-gq6dc" (UID: "07ebf665-177c-48e7-969c-28057a4ff1a4") : configmap references non-existent config key: service-ca.crt
Nov 04 01:34:38.253798 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:38.253651    5039 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 01:34:38.253798 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:38.253693    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls podName:928e6807-f385-42a2-89d5-e9ed3b1089cf nodeName:}" failed. No retries permitted until 2026-11-04 01:34:46.253684906 +0000 UTC m=+16.969871766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-m4wh8" (UID: "928e6807-f385-42a2-89d5-e9ed3b1089cf") : secret "samples-operator-tls" not found
Nov 04 01:34:38.257516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.257486    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10bc58fa-af70-41c2-9e8c-0cf7774b67fe-metrics-certs\") pod \"network-metrics-daemon-2w6dn\" (UID: \"10bc58fa-af70-41c2-9e8c-0cf7774b67fe\") " pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:38.259453 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.259434    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ebf665-177c-48e7-969c-28057a4ff1a4-metrics-certs\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:38.264328 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.264307    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2w6dn"
Nov 04 01:34:38.265195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.265177    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-sdcdd\" (UID: \"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:38.348329 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.348178    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:38.354273 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.354205    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:38.354393 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:38.354361    5039 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:38.354448 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:38.354435    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls podName:909efb32-95fc-4b85-bf58-949637c49c16 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:46.354415285 +0000 UTC m=+17.070602125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xjwj" (UID: "909efb32-95fc-4b85-bf58-949637c49c16") : secret "insights-runtime-extractor-tls" not found
Nov 04 01:34:38.375105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.375081    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-56trq"
Nov 04 01:34:38.430058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.430027    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:38.430394 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.430382    5039 scope.go:122] "RemoveContainer" containerID="cddec82cb38bf22d014e294887ef48d0b99d607c26fa4fdb895362e956eeaf5e"
Nov 04 01:34:38.430609 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:38.430590    5039 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wsc5z_openshift-console-operator(b546556a-5219-406b-88d0-a3b490c7b259)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259"
Nov 04 01:34:38.559298 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.559237    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"
Nov 04 01:34:38.665786 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.665757    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2w6dn"]
Nov 04 01:34:38.668796 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:38.668774    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10bc58fa_af70_41c2_9e8c_0cf7774b67fe.slice/crio-d6c8f5ff8cd2af69ea93801d19cb44cb8c09c2c3c99cc734df34d79e0cb79e15 WatchSource:0}: Error finding container d6c8f5ff8cd2af69ea93801d19cb44cb8c09c2c3c99cc734df34d79e0cb79e15: Status 404 returned error can't find the container with id d6c8f5ff8cd2af69ea93801d19cb44cb8c09c2c3c99cc734df34d79e0cb79e15
Nov 04 01:34:38.941138 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:38.941108    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-sdcdd"]
Nov 04 01:34:38.944675 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:38.944642    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f9ae93_ef44_48e4_8a07_12ad65a6ff0e.slice/crio-4cd9233f690bf2bbe12bc5d64a22eba54da3c19021acc1c1e9fdd0dae776d978 WatchSource:0}: Error finding container 4cd9233f690bf2bbe12bc5d64a22eba54da3c19021acc1c1e9fdd0dae776d978: Status 404 returned error can't find the container with id 4cd9233f690bf2bbe12bc5d64a22eba54da3c19021acc1c1e9fdd0dae776d978
Nov 04 01:34:39.351872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.351845    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:39.547960 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.547921    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd" event={"ID":"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e","Type":"ContainerStarted","Data":"4cd9233f690bf2bbe12bc5d64a22eba54da3c19021acc1c1e9fdd0dae776d978"}
Nov 04 01:34:39.549069 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.549033    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2w6dn" event={"ID":"10bc58fa-af70-41c2-9e8c-0cf7774b67fe","Type":"ContainerStarted","Data":"d6c8f5ff8cd2af69ea93801d19cb44cb8c09c2c3c99cc734df34d79e0cb79e15"}
Nov 04 01:34:39.600054 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.600027    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:39.615524 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.615440    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:39.615744 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.615732    5039 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:39.849016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:39.848984    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:40.096223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:40.096191    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:40.340780 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:40.340744    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:40.588981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:40.588947    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:41.557662 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:41.557619    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-sdcdd" event={"ID":"f1f9ae93-ef44-48e4-8a07-12ad65a6ff0e","Type":"ContainerStarted","Data":"c176681eec73aaaa833b16cf105fd58d017b1c4a439ad3db73ab8868fc46e2d0"}
Nov 04 01:34:41.559315 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:41.559291    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2w6dn" event={"ID":"10bc58fa-af70-41c2-9e8c-0cf7774b67fe","Type":"ContainerStarted","Data":"38d49a2554f7f052b50ca625a3db642de7a7622b066c609de5e4824c0549aad3"}
Nov 04 01:34:41.559425 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:41.559334    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2w6dn" event={"ID":"10bc58fa-af70-41c2-9e8c-0cf7774b67fe","Type":"ContainerStarted","Data":"64fc76e5743124c42397a5af1078c2122b3caa47ca502fd0ec9bf168f1c536e8"}
Nov 04 01:34:42.113313 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.113276    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgrx2"
Nov 04 01:34:42.399917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.399844    5039 scope.go:122] "RemoveContainer" containerID="cddec82cb38bf22d014e294887ef48d0b99d607c26fa4fdb895362e956eeaf5e"
Nov 04 01:34:42.564850 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.564823    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:34:42.565226 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.564935    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" event={"ID":"b546556a-5219-406b-88d0-a3b490c7b259","Type":"ContainerStarted","Data":"ae5c7f30d4dd1fef4f105764750db7d3c9dbc56ffcf3e2506747e4095cac89e1"}
Nov 04 01:34:42.565348 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.565327    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:42.566456 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.566432    5039 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-wsc5z container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.15:8443/readyz\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body=
Nov 04 01:34:42.566558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:42.566471    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z" podUID="b546556a-5219-406b-88d0-a3b490c7b259" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.15:8443/readyz\": dial tcp 10.132.0.15:8443: connect: connection refused"
Nov 04 01:34:43.572401 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:43.572371    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wsc5z"
Nov 04 01:34:43.995267 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:43.995232    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:44.242520 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:44.242480    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:44.482957 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:44.482929    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:44.535604 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:44.535561    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jn9bb"
Nov 04 01:34:44.975746 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:44.975711    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:45.404832 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:45.404802    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:45.651646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:45.651608    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:46.332652 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.332617    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:46.332652 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.332656    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:46.333358 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.333335    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07ebf665-177c-48e7-969c-28057a4ff1a4-service-ca-bundle\") pod \"router-default-54848578cf-gq6dc\" (UID: \"07ebf665-177c-48e7-969c-28057a4ff1a4\") " pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:46.335319 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.335294    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/928e6807-f385-42a2-89d5-e9ed3b1089cf-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-m4wh8\" (UID: \"928e6807-f385-42a2-89d5-e9ed3b1089cf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:46.354261 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.354234    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:46.354392 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.354331    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"
Nov 04 01:34:46.433464 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.433433    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:46.438621 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.437768    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/909efb32-95fc-4b85-bf58-949637c49c16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xjwj\" (UID: \"909efb32-95fc-4b85-bf58-949637c49c16\") " pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:46.666589 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.666546    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xjwj"
Nov 04 01:34:46.806466 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.806425    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54848578cf-gq6dc"]
Nov 04 01:34:46.809328 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:46.809299    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ebf665_177c_48e7_969c_28057a4ff1a4.slice/crio-682ae013cd98ca202cf3d8589af0c404f98ed5b1507b7708bb69272034e78620 WatchSource:0}: Error finding container 682ae013cd98ca202cf3d8589af0c404f98ed5b1507b7708bb69272034e78620: Status 404 returned error can't find the container with id 682ae013cd98ca202cf3d8589af0c404f98ed5b1507b7708bb69272034e78620
Nov 04 01:34:46.855957 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:46.855924    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8"]
Nov 04 01:34:47.061185 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:47.061114    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xjwj"]
Nov 04 01:34:47.066259 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:47.066223    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909efb32_95fc_4b85_bf58_949637c49c16.slice/crio-3bc87b5c853e8b4e4d86bea371417f9406e8bc7e004f6cf1476a22bb2286d3d1 WatchSource:0}: Error finding container 3bc87b5c853e8b4e4d86bea371417f9406e8bc7e004f6cf1476a22bb2286d3d1: Status 404 returned error can't find the container with id 3bc87b5c853e8b4e4d86bea371417f9406e8bc7e004f6cf1476a22bb2286d3d1
Nov 04 01:34:47.584752 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:47.584720    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xjwj" event={"ID":"909efb32-95fc-4b85-bf58-949637c49c16","Type":"ContainerStarted","Data":"f01e7138c7e81e7aeacafd3d18de7faa6e045a34ae2fbea0776abae4f959753e"}
Nov 04 01:34:47.585190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:47.584773    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xjwj" event={"ID":"909efb32-95fc-4b85-bf58-949637c49c16","Type":"ContainerStarted","Data":"3bc87b5c853e8b4e4d86bea371417f9406e8bc7e004f6cf1476a22bb2286d3d1"}
Nov 04 01:34:47.585872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:47.585847    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8" event={"ID":"928e6807-f385-42a2-89d5-e9ed3b1089cf","Type":"ContainerStarted","Data":"977e0b71e51cfa30668d19bf31592ef418f98badea3803ad760aa3e24640d6c0"}
Nov 04 01:34:47.587362 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:47.587340    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54848578cf-gq6dc" event={"ID":"07ebf665-177c-48e7-969c-28057a4ff1a4","Type":"ContainerStarted","Data":"42779200ef596c9b18a3f4e1b0bb7a3defc975c96e1203e5840467aa4d820cbd"}
Nov 04 01:34:47.587457 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:47.587374    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54848578cf-gq6dc" event={"ID":"07ebf665-177c-48e7-969c-28057a4ff1a4","Type":"ContainerStarted","Data":"682ae013cd98ca202cf3d8589af0c404f98ed5b1507b7708bb69272034e78620"}
Nov 04 01:34:48.287027 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:48.286975    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:48.354974 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:48.354936    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:48.358107 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:48.358084    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:48.528987 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:48.528951    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:48.591542 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:48.591519    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:48.592769 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:48.592752    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54848578cf-gq6dc"
Nov 04 01:34:49.596322 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.596287    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-6fqcs-tjdmr"]
Nov 04 01:34:49.636223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.636176    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xjwj" event={"ID":"909efb32-95fc-4b85-bf58-949637c49c16","Type":"ContainerStarted","Data":"b163ff6481dd243a4ff18bfc6db8c3766e8ff8169be9e9c453f2928cf8994cd3"}
Nov 04 01:34:49.636366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.636296    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.682245 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.682213    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"]
Nov 04 01:34:49.709126 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.709093    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-9lmfh"]
Nov 04 01:34:49.709258 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.709226    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:49.723183 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.723155    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:49.731664 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.731637    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-6fqcs-tjdmr"]
Nov 04 01:34:49.732020 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.732004    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d844f87d4-6hn9v"]
Nov 04 01:34:49.737703 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.737679    5039 patch_prober.go:29] interesting pod/image-registry-d844f87d4-6hn9v container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 01:34:49.737836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.737728    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v" podUID="038b383d-62ae-45da-a1a5-dc2246a5d946" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:34:49.761143 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.761110    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.761322 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.761220    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.761379 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.761344    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w7zn\" (UniqueName: \"kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.761379 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.761371    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.765674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.765649    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 04 01:34:49.785465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.785383    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bbc67667c-rt8nq"]
Nov 04 01:34:49.788760 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.788726    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 01:34:49.788760 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.788738    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 01:34:49.801214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.801189    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"]
Nov 04 01:34:49.801361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.801339    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.811765 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.811742    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-9lmfh"]
Nov 04 01:34:49.861868 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.861827    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w7zn\" (UniqueName: \"kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.861868 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.861867    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.862115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.861911    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85ade3b5-bd2b-4417-9d67-1e9820cf21d1-tmp\") pod \"downloads-6dd789c8b7-9lmfh\" (UID: \"85ade3b5-bd2b-4417-9d67-1e9820cf21d1\") " pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:49.862115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.861952    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.862115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.862031    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjv27\" (UniqueName: \"kubernetes.io/projected/85ade3b5-bd2b-4417-9d67-1e9820cf21d1-kube-api-access-sjv27\") pod \"downloads-6dd789c8b7-9lmfh\" (UID: \"85ade3b5-bd2b-4417-9d67-1e9820cf21d1\") " pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:49.862115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.862083    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.862115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.862112    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2fbdf966-af9e-4705-98bc-94af9a17c460-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-cz8v4\" (UID: \"2fbdf966-af9e-4705-98bc-94af9a17c460\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:49.862495 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.862463    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.863122 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.863101    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.863397 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.863374    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bbc67667c-rt8nq"]
Nov 04 01:34:49.863642 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.863625    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.865363 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.865330    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-t4nj6\""
Nov 04 01:34:49.865363 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.865339    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6fkqz\""
Nov 04 01:34:49.904227 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.904197    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:49.928047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.928010    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w7zn\" (UniqueName: \"kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn\") pod \"periodic-gathering-6fqcs-tjdmr\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") " pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:49.962912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.962868    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjv27\" (UniqueName: \"kubernetes.io/projected/85ade3b5-bd2b-4417-9d67-1e9820cf21d1-kube-api-access-sjv27\") pod \"downloads-6dd789c8b7-9lmfh\" (UID: \"85ade3b5-bd2b-4417-9d67-1e9820cf21d1\") " pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:49.963082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.962923    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ccb51c52-3f22-4465-9a26-c10b5520bcfa-installation-pull-secrets\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.962965    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2fbdf966-af9e-4705-98bc-94af9a17c460-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-cz8v4\" (UID: \"2fbdf966-af9e-4705-98bc-94af9a17c460\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:49.963082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.962993    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-bound-sa-token\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963240 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963202    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-registry-tls\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963276 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963256    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ccb51c52-3f22-4465-9a26-c10b5520bcfa-trusted-ca\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963354    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjxt\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-kube-api-access-ptjxt\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963485 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963463    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ccb51c52-3f22-4465-9a26-c10b5520bcfa-ca-trust-extracted\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963540 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963527    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85ade3b5-bd2b-4417-9d67-1e9820cf21d1-tmp\") pod \"downloads-6dd789c8b7-9lmfh\" (UID: \"85ade3b5-bd2b-4417-9d67-1e9820cf21d1\") " pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:49.963625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963557    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ccb51c52-3f22-4465-9a26-c10b5520bcfa-registry-certificates\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963619    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ccb51c52-3f22-4465-9a26-c10b5520bcfa-image-registry-private-configuration\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:49.963947 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.963927    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85ade3b5-bd2b-4417-9d67-1e9820cf21d1-tmp\") pod \"downloads-6dd789c8b7-9lmfh\" (UID: \"85ade3b5-bd2b-4417-9d67-1e9820cf21d1\") " pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:49.965872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.965850    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2fbdf966-af9e-4705-98bc-94af9a17c460-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-cz8v4\" (UID: \"2fbdf966-af9e-4705-98bc-94af9a17c460\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:49.970611 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:49.970555    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:34:50.024061 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.023991    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjv27\" (UniqueName: \"kubernetes.io/projected/85ade3b5-bd2b-4417-9d67-1e9820cf21d1-kube-api-access-sjv27\") pod \"downloads-6dd789c8b7-9lmfh\" (UID: \"85ade3b5-bd2b-4417-9d67-1e9820cf21d1\") " pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:50.056851 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.056476    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064387    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-registry-tls\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064430    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ccb51c52-3f22-4465-9a26-c10b5520bcfa-trusted-ca\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064481    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjxt\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-kube-api-access-ptjxt\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064514    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ccb51c52-3f22-4465-9a26-c10b5520bcfa-ca-trust-extracted\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064557    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ccb51c52-3f22-4465-9a26-c10b5520bcfa-registry-certificates\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064652    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ccb51c52-3f22-4465-9a26-c10b5520bcfa-image-registry-private-configuration\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064680    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ccb51c52-3f22-4465-9a26-c10b5520bcfa-installation-pull-secrets\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.064712    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-bound-sa-token\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.066125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.066024    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ccb51c52-3f22-4465-9a26-c10b5520bcfa-ca-trust-extracted\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.068408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.067656    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ccb51c52-3f22-4465-9a26-c10b5520bcfa-trusted-ca\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.068408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.067676    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ccb51c52-3f22-4465-9a26-c10b5520bcfa-registry-certificates\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.072786 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.072682    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:34:50.078133 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.077733    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ccb51c52-3f22-4465-9a26-c10b5520bcfa-image-registry-private-configuration\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.078133 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.078101    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-registry-tls\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.079946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.079900    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ccb51c52-3f22-4465-9a26-c10b5520bcfa-installation-pull-secrets\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.130108 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.130054    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjxt\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-kube-api-access-ptjxt\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.130519 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.130481    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccb51c52-3f22-4465-9a26-c10b5520bcfa-bound-sa-token\") pod \"image-registry-bbc67667c-rt8nq\" (UID: \"ccb51c52-3f22-4465-9a26-c10b5520bcfa\") " pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.148459 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.148406    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:50.376960 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.376927    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-6fqcs-tjdmr"]
Nov 04 01:34:50.381660 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:50.381628    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614a9729_7186_4c8c_8426_0059050302da.slice/crio-fd35d1e978d077a50a9ac7e9108aaaebe55c3610e1ef6dd81b6bd44cf19b21a1 WatchSource:0}: Error finding container fd35d1e978d077a50a9ac7e9108aaaebe55c3610e1ef6dd81b6bd44cf19b21a1: Status 404 returned error can't find the container with id fd35d1e978d077a50a9ac7e9108aaaebe55c3610e1ef6dd81b6bd44cf19b21a1
Nov 04 01:34:50.423933 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.423909    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:50.462169 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.462143    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"]
Nov 04 01:34:50.466169 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:50.466135    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbdf966_af9e_4705_98bc_94af9a17c460.slice/crio-8e20096bfefd9814ad53edd55fad3c2a2f56d62f4afa710aea7fe2f4bbcea72d WatchSource:0}: Error finding container 8e20096bfefd9814ad53edd55fad3c2a2f56d62f4afa710aea7fe2f4bbcea72d: Status 404 returned error can't find the container with id 8e20096bfefd9814ad53edd55fad3c2a2f56d62f4afa710aea7fe2f4bbcea72d
Nov 04 01:34:50.531697 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.531660    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-9lmfh"]
Nov 04 01:34:50.605960 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.605925    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-9lmfh" event={"ID":"85ade3b5-bd2b-4417-9d67-1e9820cf21d1","Type":"ContainerStarted","Data":"5d78a784cb60d58b67c09dbf3506b6309b93d7fe6f2f0f04a5a566422e2e5111"}
Nov 04 01:34:50.608065 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.608035    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8" event={"ID":"928e6807-f385-42a2-89d5-e9ed3b1089cf","Type":"ContainerStarted","Data":"e0b31aa83c85031687ba1f4bfb1983f796cc9a3b495b8fe1e8d264df9578599e"}
Nov 04 01:34:50.608182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.608087    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-m4wh8" event={"ID":"928e6807-f385-42a2-89d5-e9ed3b1089cf","Type":"ContainerStarted","Data":"babd58c9567429080234da82cbd67b1d39137adcac421d916b93ff50361f4891"}
Nov 04 01:34:50.609610 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.609581    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4" event={"ID":"2fbdf966-af9e-4705-98bc-94af9a17c460","Type":"ContainerStarted","Data":"8e20096bfefd9814ad53edd55fad3c2a2f56d62f4afa710aea7fe2f4bbcea72d"}
Nov 04 01:34:50.611677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.611615    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr" event={"ID":"614a9729-7186-4c8c-8426-0059050302da","Type":"ContainerStarted","Data":"c31f2f834b819b0efa722724194bcb2321c7fbc3b1f65b61f5ac68536029d5da"}
Nov 04 01:34:50.611677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.611657    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr" event={"ID":"614a9729-7186-4c8c-8426-0059050302da","Type":"ContainerStarted","Data":"fd35d1e978d077a50a9ac7e9108aaaebe55c3610e1ef6dd81b6bd44cf19b21a1"}
Nov 04 01:34:50.837365 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.837278    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bbc67667c-rt8nq"]
Nov 04 01:34:50.840371 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:50.839766    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr" podStartSLOduration=1.839746528 podStartE2EDuration="1.839746528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:50.839009455 +0000 UTC m=+21.555196312" watchObservedRunningTime="2026-11-04 01:34:50.839746528 +0000 UTC m=+21.555933431"
Nov 04 01:34:50.842054 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:50.842021    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb51c52_3f22_4465_9a26_c10b5520bcfa.slice/crio-d6fccf2a691866b29d9436f3218ba8e5cc680e027e10d2eb4cae5f2ec7663c6e WatchSource:0}: Error finding container d6fccf2a691866b29d9436f3218ba8e5cc680e027e10d2eb4cae5f2ec7663c6e: Status 404 returned error can't find the container with id d6fccf2a691866b29d9436f3218ba8e5cc680e027e10d2eb4cae5f2ec7663c6e
Nov 04 01:34:51.493217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.493186    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:51.617894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.617780    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq" event={"ID":"ccb51c52-3f22-4465-9a26-c10b5520bcfa","Type":"ContainerStarted","Data":"c8a7e80e4ba4de9550528b473ed4e3eef296cc3ec61fbb5efe22fa34b6077f77"}
Nov 04 01:34:51.617894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.617872    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq" event={"ID":"ccb51c52-3f22-4465-9a26-c10b5520bcfa","Type":"ContainerStarted","Data":"d6fccf2a691866b29d9436f3218ba8e5cc680e027e10d2eb4cae5f2ec7663c6e"}
Nov 04 01:34:51.618486 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.617932    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:34:51.622339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.622314    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xjwj" event={"ID":"909efb32-95fc-4b85-bf58-949637c49c16","Type":"ContainerStarted","Data":"4fac596077062b2b1f9f210ec2fdc5e58ea7e644325397cc366281b470a4487a"}
Nov 04 01:34:51.740075 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.740012    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq" podStartSLOduration=2.739991351 podStartE2EDuration="2.739991351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:51.738313142 +0000 UTC m=+22.454499997" watchObservedRunningTime="2026-11-04 01:34:51.739991351 +0000 UTC m=+22.456178208"
Nov 04 01:34:51.745669 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:51.745643    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:52.379867 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.379544    5039 patch_prober.go:29] interesting pod/image-registry-557fcb4957-k6rbt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 01:34:52.379867 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.379619    5039 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt" podUID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:34:52.634372 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.634329    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4" event={"ID":"2fbdf966-af9e-4705-98bc-94af9a17c460","Type":"ContainerStarted","Data":"d2b608a4bb910a97d18f76bb1630a0abda56826fd9955cd01659ac33a7813195"}
Nov 04 01:34:52.634845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.634825    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:52.640404 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.640378    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4"
Nov 04 01:34:52.749358 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.749299    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-cz8v4" podStartSLOduration=2.02805969 podStartE2EDuration="3.749285528s" totalImagesPullingTime="1.721225838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:52.74816825 +0000 UTC m=+23.464355105" watchObservedRunningTime="2026-11-04 01:34:52.749285528 +0000 UTC m=+23.465472383"
Nov 04 01:34:52.755743 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:34:52.755647    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:34:52.946537 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:52.946463    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jn9bb_d3b79cfe-4f1c-4786-9f97-e87ce36cbba8/dns/0.log"
Nov 04 01:34:53.149624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:53.149588    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jn9bb_d3b79cfe-4f1c-4786-9f97-e87ce36cbba8/kube-rbac-proxy/0.log"
Nov 04 01:34:53.546722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:53.546686    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzccz_e13536d8-6703-46a7-9e8e-8136e1ff8979/dns-node-resolver/0.log"
Nov 04 01:34:53.769122 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:53.769056    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:53.947411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:53.947379    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-557fcb4957-k6rbt_cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/registry/0.log"
Nov 04 01:34:54.022273 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:54.022234    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:54.147101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:54.147073    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bbc67667c-rt8nq_ccb51c52-3f22-4465-9a26-c10b5520bcfa/registry/0.log"
Nov 04 01:34:54.269228 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:54.269104    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:54.347040 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:54.347011    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d844f87d4-6hn9v_038b383d-62ae-45da-a1a5-dc2246a5d946/registry/0.log"
Nov 04 01:34:54.512678 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:54.512648    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:54.551984 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:54.551917    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kl2jb_be3ebfff-ac2e-4429-ac1e-1fb87d7599d0/node-ca/0.log"
Nov 04 01:34:55.146432 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:55.146407    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54848578cf-gq6dc_07ebf665-177c-48e7-969c-28057a4ff1a4/router/0.log"
Nov 04 01:34:55.546844 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:55.546780    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m66qn_d4f41073-2c64-44f1-9153-ca4e0e2c0b50/serve-healthcheck-canary/0.log"
Nov 04 01:34:57.544556 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:57.544528    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:34:58.252125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:58.252088    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:58.506555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:58.506467    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:58.978154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:58.978114    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:59.229097 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.229021    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:59.489414 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.489340    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:59.624222 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.624192    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cvcjq"]
Nov 04 01:34:59.627174 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.627152    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.730004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.729971    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 01:34:59.730004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.729992    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 01:34:59.731075 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.731040    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:59.738115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.738092    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:34:59.759364 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759301    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvx9h\" (UniqueName: \"kubernetes.io/projected/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-kube-api-access-bvx9h\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759364 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759342    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-textfile\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759526 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759436    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759526 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759512    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-root\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759542    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-tls\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759610    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759694    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-wtmp\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759724    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-sys\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.759798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.759777    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-metrics-client-ca\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.779644 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.779616    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 01:34:59.779771 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.779651    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 01:34:59.779771 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.779662    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bnxvr\""
Nov 04 01:34:59.860302 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860268    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-wtmp\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860315    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-sys\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860344    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-metrics-client-ca\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860410    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvx9h\" (UniqueName: \"kubernetes.io/projected/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-kube-api-access-bvx9h\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860439    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-textfile\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860486    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860436    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-wtmp\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860536    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-root\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860562    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-tls\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860608    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860774    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-root\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.860856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860437    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-sys\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.861008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.860976    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-textfile\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.861127 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.861118    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-metrics-client-ca\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.861178 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.861115    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.863265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.863240    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-tls\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.863465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.863433    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.921166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.921138    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvx9h\" (UniqueName: \"kubernetes.io/projected/2256d437-8a82-4a72-acc4-3fc5cb85c9a2-kube-api-access-bvx9h\") pod \"node-exporter-cvcjq\" (UID: \"2256d437-8a82-4a72-acc4-3fc5cb85c9a2\") " pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.944303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.944267    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cvcjq"
Nov 04 01:34:59.956924 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:34:59.956891    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2256d437_8a82_4a72_acc4_3fc5cb85c9a2.slice/crio-1cbcc5c9293ef774985ae92680590e4b0bb0bcc60a11dc9dd93bb70794b4c910 WatchSource:0}: Error finding container 1cbcc5c9293ef774985ae92680590e4b0bb0bcc60a11dc9dd93bb70794b4c910: Status 404 returned error can't find the container with id 1cbcc5c9293ef774985ae92680590e4b0bb0bcc60a11dc9dd93bb70794b4c910
Nov 04 01:34:59.977992 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:34:59.977962    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:00.230559 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:00.230529    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:00.483101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:00.483014    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:00.676615 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:00.676556    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvcjq" event={"ID":"2256d437-8a82-4a72-acc4-3fc5cb85c9a2","Type":"ContainerStarted","Data":"1cbcc5c9293ef774985ae92680590e4b0bb0bcc60a11dc9dd93bb70794b4c910"}
Nov 04 01:35:01.049026 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:01.048987    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:01.301905 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:01.301821    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:01.683995 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:01.683961    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvcjq" event={"ID":"2256d437-8a82-4a72-acc4-3fc5cb85c9a2","Type":"ContainerStarted","Data":"4ec0db19f092f05b4fbf6b2dfa1dec23b63824210b330cf25a2dca178737c8c5"}
Nov 04 01:35:01.918900 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:01.918867    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:02.691381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:02.691340    5039 generic.go:356] "Generic (PLEG): container finished" podID="2256d437-8a82-4a72-acc4-3fc5cb85c9a2" containerID="4ec0db19f092f05b4fbf6b2dfa1dec23b63824210b330cf25a2dca178737c8c5" exitCode=0
Nov 04 01:35:02.691900 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:02.691398    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvcjq" event={"ID":"2256d437-8a82-4a72-acc4-3fc5cb85c9a2","Type":"ContainerDied","Data":"4ec0db19f092f05b4fbf6b2dfa1dec23b63824210b330cf25a2dca178737c8c5"}
Nov 04 01:35:02.691900 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:02.691465    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvcjq" event={"ID":"2256d437-8a82-4a72-acc4-3fc5cb85c9a2","Type":"ContainerStarted","Data":"17d9631d27be350cb0b3bbe70534c8eeee838beed0bd8d34f721099669e7ec7c"}
Nov 04 01:35:02.691900 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:02.691531    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cvcjq" event={"ID":"2256d437-8a82-4a72-acc4-3fc5cb85c9a2","Type":"ContainerStarted","Data":"78038e628909661a4f182306ed32c9dac523eb6fdedb09f5fc2537049de85e74"}
Nov 04 01:35:02.757126 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:02.757098    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-68r2l"
Nov 04 01:35:02.810758 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:02.810553    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cvcjq" podStartSLOduration=2.191084021 podStartE2EDuration="3.810538356s" totalImagesPullingTime="1.532183966s" totalInitContainerRuntime="87.270369ms" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:02.809655105 +0000 UTC m=+33.525841960" watchObservedRunningTime="2026-11-04 01:35:02.810538356 +0000 UTC m=+33.526725212"
Nov 04 01:35:03.882336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:03.882304    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:04.131016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:04.130976    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:04.390021 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:04.389986    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:04.636595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:04.636532    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:06.714902 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:06.714811    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-9lmfh" event={"ID":"85ade3b5-bd2b-4417-9d67-1e9820cf21d1","Type":"ContainerStarted","Data":"e33784f3d277406a286db8472af35df2b896fc3b1d880427b841cdd12a98df2a"}
Nov 04 01:35:06.715325 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:06.714983    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:35:06.734687 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:06.734654    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-9lmfh"
Nov 04 01:35:06.931147 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:06.931078    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-9lmfh" podStartSLOduration=2.102102889 podStartE2EDuration="17.931058684s" totalImagesPullingTime="15.828955795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:06.880342862 +0000 UTC m=+37.596529718" watchObservedRunningTime="2026-11-04 01:35:06.931058684 +0000 UTC m=+37.647245539"
Nov 04 01:35:08.901235 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:08.901195    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:12.062247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:12.062160    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:12.309789 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:12.309747    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:12.640031 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:12.640001    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bbc67667c-rt8nq"
Nov 04 01:35:12.865946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:12.865914    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557fcb4957-k6rbt"]
Nov 04 01:35:13.071728 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:13.071651    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:13.321679 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:13.321649    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:13.571375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:13.571334    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:13.907353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:13.907322    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:14.151912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:14.151880    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:14.756906 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:14.756855    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v" podUID="038b383d-62ae-45da-a1a5-dc2246a5d946" containerName="registry" containerID="cri-o://fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071" gracePeriod=30
Nov 04 01:35:15.040247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.040219    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:35:15.121659 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121632    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.121826 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121698    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.121826 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121731    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.121826 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121765    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.121826 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121810    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.122012 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121839    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.122012 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121872    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.122012 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.121902    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") pod \"038b383d-62ae-45da-a1a5-dc2246a5d946\" (UID: \"038b383d-62ae-45da-a1a5-dc2246a5d946\") "
Nov 04 01:35:15.122539 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.122466    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:15.122772 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.122749    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:15.124602 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.124547    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:15.124722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.124629    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:15.124722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.124633    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:15.124722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.124689    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "kube-api-access-tw5xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:15.124722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.124708    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:15.128866 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.128838    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted" pod "038b383d-62ae-45da-a1a5-dc2246a5d946" (UID: "038b383d-62ae-45da-a1a5-dc2246a5d946"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:15.222499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222466    5039 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222499    5039 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-bound-sa-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222928 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222512    5039 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/038b383d-62ae-45da-a1a5-dc2246a5d946-ca-trust-extracted\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222928 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222525    5039 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-image-registry-private-configuration\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222928 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222539    5039 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-registry-certificates\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222928 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222552    5039 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/038b383d-62ae-45da-a1a5-dc2246a5d946-trusted-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222928 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222579    5039 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/038b383d-62ae-45da-a1a5-dc2246a5d946-installation-pull-secrets\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.222928 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.222592    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw5xq\" (UniqueName: \"kubernetes.io/projected/038b383d-62ae-45da-a1a5-dc2246a5d946-kube-api-access-tw5xq\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:15.764660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.764557    5039 generic.go:356] "Generic (PLEG): container finished" podID="038b383d-62ae-45da-a1a5-dc2246a5d946" containerID="fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071" exitCode=0
Nov 04 01:35:15.764660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.764626    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v"
Nov 04 01:35:15.764660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.764641    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v" event={"ID":"038b383d-62ae-45da-a1a5-dc2246a5d946","Type":"ContainerDied","Data":"fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071"}
Nov 04 01:35:15.764909 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.764695    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d844f87d4-6hn9v" event={"ID":"038b383d-62ae-45da-a1a5-dc2246a5d946","Type":"ContainerDied","Data":"cb3d25ea3435d6c2922d2cb027cd9eddb84c01846adf116859911fc73a244450"}
Nov 04 01:35:15.764909 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.764715    5039 scope.go:122] "RemoveContainer" containerID="fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071"
Nov 04 01:35:15.773837 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.773820    5039 scope.go:122] "RemoveContainer" containerID="fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071"
Nov 04 01:35:15.774155 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:35:15.774127    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071\": container with ID starting with fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071 not found: ID does not exist" containerID="fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071"
Nov 04 01:35:15.774260 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.774163    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071"} err="failed to get container status \"fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071\": rpc error: code = NotFound desc = could not find container \"fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071\": container with ID starting with fd8f3e7f2d1b0a6f559634427494e89a3d254dbcd6176e6abf3ba6b73ab2d071 not found: ID does not exist"
Nov 04 01:35:15.976269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:15.976232    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d844f87d4-6hn9v"]
Nov 04 01:35:16.025501 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:16.025431    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d844f87d4-6hn9v"]
Nov 04 01:35:17.404216 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:17.404179    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="038b383d-62ae-45da-a1a5-dc2246a5d946" path="/var/lib/kubelet/pods/038b383d-62ae-45da-a1a5-dc2246a5d946/volumes"
Nov 04 01:35:18.302954 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:18.302923    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:18.543939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:18.543904    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:19.211431 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:19.211399    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:19.453405 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:19.453377    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:19.723252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:19.723216    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:19.991824 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:19.991747    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:20.417649 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:20.417617    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:20.665001 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:20.664972    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:21.979339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:21.979304    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-ttvxq-rrlpl"]
Nov 04 01:35:21.979853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:21.979562    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl" podUID="4164bd42-81bf-478d-8409-001aed0783dc" containerName="insights-gathering" containerID="cri-o://7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991" gracePeriod=30
Nov 04 01:35:22.294874 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.294850    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:35:22.386305 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386274    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") pod \"4164bd42-81bf-478d-8409-001aed0783dc\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") "
Nov 04 01:35:22.386305 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386312    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") pod \"4164bd42-81bf-478d-8409-001aed0783dc\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") "
Nov 04 01:35:22.386516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386334    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") pod \"4164bd42-81bf-478d-8409-001aed0783dc\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") "
Nov 04 01:35:22.386516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386353    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") pod \"4164bd42-81bf-478d-8409-001aed0783dc\" (UID: \"4164bd42-81bf-478d-8409-001aed0783dc\") "
Nov 04 01:35:22.386721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386696    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path" pod "4164bd42-81bf-478d-8409-001aed0783dc" (UID: "4164bd42-81bf-478d-8409-001aed0783dc"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:22.386759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386722    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle" pod "4164bd42-81bf-478d-8409-001aed0783dc" (UID: "4164bd42-81bf-478d-8409-001aed0783dc"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:22.386792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.386749    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle" pod "4164bd42-81bf-478d-8409-001aed0783dc" (UID: "4164bd42-81bf-478d-8409-001aed0783dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:22.392472 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.392420    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx" pod "4164bd42-81bf-478d-8409-001aed0783dc" (UID: "4164bd42-81bf-478d-8409-001aed0783dc"). InnerVolumeSpecName "kube-api-access-dpwrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:22.487820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.487785    5039 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4164bd42-81bf-478d-8409-001aed0783dc-archives-path\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:22.487820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.487813    5039 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-service-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:22.487820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.487822    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpwrx\" (UniqueName: \"kubernetes.io/projected/4164bd42-81bf-478d-8409-001aed0783dc-kube-api-access-dpwrx\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:22.488051 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.487833    5039 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4164bd42-81bf-478d-8409-001aed0783dc-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:22.750486 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:35:22.750446    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:35:22.794658 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.794626    5039 generic.go:356] "Generic (PLEG): container finished" podID="4164bd42-81bf-478d-8409-001aed0783dc" containerID="7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991" exitCode=2
Nov 04 01:35:22.794832 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.794668    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl"
Nov 04 01:35:22.794832 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.794712    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl" event={"ID":"4164bd42-81bf-478d-8409-001aed0783dc","Type":"ContainerDied","Data":"7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991"}
Nov 04 01:35:22.794832 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.794761    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-ttvxq-rrlpl" event={"ID":"4164bd42-81bf-478d-8409-001aed0783dc","Type":"ContainerDied","Data":"de5b95e01eded23af551241035e64c98ed4b07d743da384de7f0ee68938b4f8a"}
Nov 04 01:35:22.794832 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.794781    5039 scope.go:122] "RemoveContainer" containerID="7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991"
Nov 04 01:35:22.803836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.803817    5039 scope.go:122] "RemoveContainer" containerID="7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991"
Nov 04 01:35:22.804117 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:35:22.804092    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991\": container with ID starting with 7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991 not found: ID does not exist" containerID="7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991"
Nov 04 01:35:22.804171 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.804128    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991"} err="failed to get container status \"7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991\": rpc error: code = NotFound desc = could not find container \"7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991\": container with ID starting with 7aa596da8e2471c19ce300ad254e3c806eb733bb2ffdfff80e3db531496aa991 not found: ID does not exist"
Nov 04 01:35:22.957248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:22.957213    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-ttvxq-rrlpl"]
Nov 04 01:35:23.006600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:23.006503    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-ttvxq-rrlpl"]
Nov 04 01:35:23.404255 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:23.404221    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4164bd42-81bf-478d-8409-001aed0783dc" path="/var/lib/kubelet/pods/4164bd42-81bf-478d-8409-001aed0783dc/volumes"
Nov 04 01:35:29.631670 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:29.631590    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:29.876076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:29.876044    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:33.300040 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:33.300006    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:33.545064 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:33.545028    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:37.908382 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:37.908337    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt" podUID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" containerName="registry" containerID="cri-o://1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d" gracePeriod=30
Nov 04 01:35:38.184662 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.184597    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:35:38.326203 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326173    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326225    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326253    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326278    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326303    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326330    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326399    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326453    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") pod \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\" (UID: \"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2\") "
Nov 04 01:35:38.326677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326642    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:38.326883 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.326731    5039 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-certificates\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.327625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.327546    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:38.329052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.329012    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:38.329225 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.329197    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:38.329342 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.329251    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:38.329404 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.329354    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "kube-api-access-ptfwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:38.329462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.329408    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:38.332835 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.332812    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted" pod "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" (UID: "cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:38.427291 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427261    5039 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-bound-sa-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.427291 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427293    5039 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-trusted-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.427506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427305    5039 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-registry-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.427506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427317    5039 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-ca-trust-extracted\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.427506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427330    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptfwf\" (UniqueName: \"kubernetes.io/projected/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-kube-api-access-ptfwf\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.427506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427343    5039 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-installation-pull-secrets\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.427506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.427357    5039 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2-image-registry-private-configuration\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:38.859099 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.859070    5039 generic.go:356] "Generic (PLEG): container finished" podID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" containerID="1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d" exitCode=0
Nov 04 01:35:38.859274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.859109    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt" event={"ID":"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2","Type":"ContainerDied","Data":"1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d"}
Nov 04 01:35:38.859274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.859151    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt" event={"ID":"cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2","Type":"ContainerDied","Data":"f4cd2e40b338bbf95de986b22de2ccb11c5f6127fde9bfc3a4d51e2fec5cdb3e"}
Nov 04 01:35:38.859274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.859166    5039 scope.go:122] "RemoveContainer" containerID="1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d"
Nov 04 01:35:38.859274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.859169    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557fcb4957-k6rbt"
Nov 04 01:35:38.868256 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.868235    5039 scope.go:122] "RemoveContainer" containerID="1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d"
Nov 04 01:35:38.868527 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:35:38.868504    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d\": container with ID starting with 1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d not found: ID does not exist" containerID="1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d"
Nov 04 01:35:38.868634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:38.868536    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d"} err="failed to get container status \"1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d\": rpc error: code = NotFound desc = could not find container \"1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d\": container with ID starting with 1ef9fbe8418f837f3ebd8e7f60e277eb83af272d35e8af98ea7be1fbc2c70c9d not found: ID does not exist"
Nov 04 01:35:38.881450 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:35:38.881421    5039 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1fd32d_06a3_4b5c_8c3c_dc6b3580b8e2.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1fd32d_06a3_4b5c_8c3c_dc6b3580b8e2.slice/crio-f4cd2e40b338bbf95de986b22de2ccb11c5f6127fde9bfc3a4d51e2fec5cdb3e\": RecentStats: unable to find data in memory cache]"
Nov 04 01:35:39.021879 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:39.021848    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557fcb4957-k6rbt"]
Nov 04 01:35:39.071441 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:39.071407    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-557fcb4957-k6rbt"]
Nov 04 01:35:39.405212 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:39.405184    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" path="/var/lib/kubelet/pods/cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2/volumes"
Nov 04 01:35:41.654672 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:41.654641    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:41.899679 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:41.899647    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:43.332437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:43.332403    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:43.734371 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:43.734337    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:45.572895 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:45.572867    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:45.819759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:45.819729    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:48.291682 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:48.291639    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:48.540978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:48.540948    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:49.903341 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:49.903299    5039 generic.go:356] "Generic (PLEG): container finished" podID="df3c83fd-42b4-423d-a2a2-9da642a21118" containerID="bf060721ec495af2cd42411cfae8ee90f277b5b04111d22e79a020d959eb4a8e" exitCode=0
Nov 04 01:35:49.903812 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:49.903382    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4" event={"ID":"df3c83fd-42b4-423d-a2a2-9da642a21118","Type":"ContainerDied","Data":"bf060721ec495af2cd42411cfae8ee90f277b5b04111d22e79a020d959eb4a8e"}
Nov 04 01:35:49.903812 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:49.903719    5039 scope.go:122] "RemoveContainer" containerID="bf060721ec495af2cd42411cfae8ee90f277b5b04111d22e79a020d959eb4a8e"
Nov 04 01:35:50.909180 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:50.909146    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-v5wz4" event={"ID":"df3c83fd-42b4-423d-a2a2-9da642a21118","Type":"ContainerStarted","Data":"922147171ea5cc9fcc9cb3b04ffbf2cb29d90d5877e5bb7abb17f5e33c420c40"}
Nov 04 01:35:52.750236 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:35:52.750199    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:35:53.857733 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:53.857698    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:54.247769 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:54.247691    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:54.522817 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:35:54.522742    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:01.108806 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:01.108770    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:01.360910 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:01.360784    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:04.802630 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:04.802590    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:05.051268 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:05.051232    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:05.301922 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:05.301889    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:10.147955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:10.147917    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:10.397049 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:10.397012    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:15.218881 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:15.218840    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:18.013731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:18.013693    5039 generic.go:356] "Generic (PLEG): container finished" podID="614a9729-7186-4c8c-8426-0059050302da" containerID="c31f2f834b819b0efa722724194bcb2321c7fbc3b1f65b61f5ac68536029d5da" exitCode=0
Nov 04 01:36:18.014153 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:18.013772    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr" event={"ID":"614a9729-7186-4c8c-8426-0059050302da","Type":"ContainerDied","Data":"c31f2f834b819b0efa722724194bcb2321c7fbc3b1f65b61f5ac68536029d5da"}
Nov 04 01:36:18.298722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:18.298644    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:18.550119 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:18.550039    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:19.148794 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.148773    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:36:19.252809 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.252773    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn\" (UniqueName: \"kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn\") pod \"614a9729-7186-4c8c-8426-0059050302da\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") "
Nov 04 01:36:19.252976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.252840    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle\") pod \"614a9729-7186-4c8c-8426-0059050302da\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") "
Nov 04 01:36:19.252976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.252867    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle\") pod \"614a9729-7186-4c8c-8426-0059050302da\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") "
Nov 04 01:36:19.252976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.252922    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path\") pod \"614a9729-7186-4c8c-8426-0059050302da\" (UID: \"614a9729-7186-4c8c-8426-0059050302da\") "
Nov 04 01:36:19.253291 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.253261    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path" pod "614a9729-7186-4c8c-8426-0059050302da" (UID: "614a9729-7186-4c8c-8426-0059050302da"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:36:19.253291 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.253271    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle" pod "614a9729-7186-4c8c-8426-0059050302da" (UID: "614a9729-7186-4c8c-8426-0059050302da"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:36:19.253291 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.253281    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle" pod "614a9729-7186-4c8c-8426-0059050302da" (UID: "614a9729-7186-4c8c-8426-0059050302da"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:36:19.255081 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.255063    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn" pod "614a9729-7186-4c8c-8426-0059050302da" (UID: "614a9729-7186-4c8c-8426-0059050302da"). InnerVolumeSpecName "kube-api-access-4w7zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:36:19.353600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.353550    5039 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/614a9729-7186-4c8c-8426-0059050302da-archives-path\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:19.353600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.353599    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w7zn\" (UniqueName: \"kubernetes.io/projected/614a9729-7186-4c8c-8426-0059050302da-kube-api-access-4w7zn\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:19.353820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.353611    5039 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-service-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:19.353820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:19.353620    5039 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614a9729-7186-4c8c-8426-0059050302da-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:20.022968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:20.022936    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr"
Nov 04 01:36:20.022968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:20.022945    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6fqcs-tjdmr" event={"ID":"614a9729-7186-4c8c-8426-0059050302da","Type":"ContainerDied","Data":"fd35d1e978d077a50a9ac7e9108aaaebe55c3610e1ef6dd81b6bd44cf19b21a1"}
Nov 04 01:36:20.023172 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:20.022988    5039 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd35d1e978d077a50a9ac7e9108aaaebe55c3610e1ef6dd81b6bd44cf19b21a1"
Nov 04 01:36:20.071620 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:20.071587    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:21.221458 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:21.221426    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:21.468243 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:21.468210    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:22.750300 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:36:22.750259    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:36:24.012957 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:24.012923    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:36.047338 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:36.047300    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:36.297850 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:36:36.297770    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:52.750778 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:36:52.750745    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:37:20.423012 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:20.422979    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:22.750254 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:37:22.750218    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:37:52.333033 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:52.333000    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:52.604588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:52.604538    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:52.750325 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:37:52.750291    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:37:53.883457 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:53.883415    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:54.137136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:54.137071    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:55.156886 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:55.156851    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:59.659796 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.659762    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"]
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660204    5039 cpu_manager.go:414] "containerMap: removing container" podUID="4164bd42-81bf-478d-8409-001aed0783dc" containerName="insights-gathering"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660221    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4164bd42-81bf-478d-8409-001aed0783dc" containerName="insights-gathering"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660240    5039 cpu_manager.go:414] "containerMap: removing container" podUID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" containerName="registry"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660248    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" containerName="registry"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660274    5039 cpu_manager.go:414] "containerMap: removing container" podUID="614a9729-7186-4c8c-8426-0059050302da" containerName="insights-gathering"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660282    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="614a9729-7186-4c8c-8426-0059050302da" containerName="insights-gathering"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660308    5039 cpu_manager.go:414] "containerMap: removing container" podUID="038b383d-62ae-45da-a1a5-dc2246a5d946" containerName="registry"
Nov 04 01:37:59.660336 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660315    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="038b383d-62ae-45da-a1a5-dc2246a5d946" containerName="registry"
Nov 04 01:37:59.660770 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660396    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="614a9729-7186-4c8c-8426-0059050302da" containerName="insights-gathering"
Nov 04 01:37:59.660770 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660411    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="038b383d-62ae-45da-a1a5-dc2246a5d946" containerName="registry"
Nov 04 01:37:59.660770 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660423    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="4164bd42-81bf-478d-8409-001aed0783dc" containerName="insights-gathering"
Nov 04 01:37:59.660770 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.660433    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf1fd32d-06a3-4b5c-8c3c-dc6b3580b8e2" containerName="registry"
Nov 04 01:37:59.662439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.662421    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.702620 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.702584    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b6td\" (UniqueName: \"kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.702763 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.702641    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.702763 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.702738    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.764076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.764052    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 01:37:59.764076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.764070    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 01:37:59.764242 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.764098    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-mpxt5\""
Nov 04 01:37:59.765159 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.765142    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"]
Nov 04 01:37:59.803524 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.803499    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b6td\" (UniqueName: \"kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.803666 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.803545    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.803666 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.803593    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.803883 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.803870    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.803949 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.803935    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.864692 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.864658    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b6td\" (UniqueName: \"kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:37:59.974192 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:37:59.974140    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:38:00.201157 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:00.201127    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:00.345436 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:00.345405    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"]
Nov 04 01:38:00.404708 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:00.404673    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97" event={"ID":"a5444637-8b12-40d2-b791-689a647e0c90","Type":"ContainerStarted","Data":"9851e833875265b9fbf3e9c93bd1ab7280d5aa84d290d3c4353152d87832dd7d"}
Nov 04 01:38:00.459600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:00.459555    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:01.411660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.411623    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"]
Nov 04 01:38:01.413909 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.413888    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.476085 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.476058    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 01:38:01.487229 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.487208    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-plld6\""
Nov 04 01:38:01.518256 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518208    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbeaf941-77d4-45f8-8e94-825323dc4a96-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518396 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518290    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518396 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518323    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518396 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518370    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518557 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518419    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518557 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518455    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518557 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518482    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518557 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518498    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.518557 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.518517    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ms8c\" (UniqueName: \"kubernetes.io/projected/dbeaf941-77d4-45f8-8e94-825323dc4a96-kube-api-access-6ms8c\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.525156 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.525133    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"]
Nov 04 01:38:01.619891 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.619861    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ms8c\" (UniqueName: \"kubernetes.io/projected/dbeaf941-77d4-45f8-8e94-825323dc4a96-kube-api-access-6ms8c\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620049 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.619906    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbeaf941-77d4-45f8-8e94-825323dc4a96-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620049 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.619933    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620049 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.619953    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620280 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620262    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620322    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620345    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620401    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620554 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620447    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620554 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620472    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620700    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbeaf941-77d4-45f8-8e94-825323dc4a96-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.620830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.620814    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.621213 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.621060    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.621324 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.621130    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.623168 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.623149    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.623245 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.623209    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.658089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.658054    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:01.680733 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.680651    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ms8c\" (UniqueName: \"kubernetes.io/projected/dbeaf941-77d4-45f8-8e94-825323dc4a96-kube-api-access-6ms8c\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.680733 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.680698    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbeaf941-77d4-45f8-8e94-825323dc4a96-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vkm88\" (UID: \"dbeaf941-77d4-45f8-8e94-825323dc4a96\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.727867 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.727842    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:01.913521 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:01.913469    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:02.158281 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:02.158246    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"]
Nov 04 01:38:02.161622 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:02.161527    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:02.161743 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:38:02.161700    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbeaf941_77d4_45f8_8e94_825323dc4a96.slice/crio-f722ece47a9fc3ed23402990210f32b0986c6046b4101740ece196e69daf1380 WatchSource:0}: Error finding container f722ece47a9fc3ed23402990210f32b0986c6046b4101740ece196e69daf1380: Status 404 returned error can't find the container with id f722ece47a9fc3ed23402990210f32b0986c6046b4101740ece196e69daf1380
Nov 04 01:38:02.414490 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:02.414414    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" event={"ID":"dbeaf941-77d4-45f8-8e94-825323dc4a96","Type":"ContainerStarted","Data":"f722ece47a9fc3ed23402990210f32b0986c6046b4101740ece196e69daf1380"}
Nov 04 01:38:05.378503 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:05.378471    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:06.227071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:06.227032    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:38:06.227186 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:06.227119    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:38:06.227186 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:06.227162    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:38:06.432589 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:06.432543    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" event={"ID":"dbeaf941-77d4-45f8-8e94-825323dc4a96","Type":"ContainerStarted","Data":"d8ae395e374e8ed4ef47d998e84093b92e69163cef17c591caeb1f45ac1109b4"}
Nov 04 01:38:06.563055 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:06.562810    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podStartSLOduration=1.5004073930000001 podStartE2EDuration="5.562787912s" totalImagesPullingTime="4.062380519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:06.562062461 +0000 UTC m=+217.278249317" watchObservedRunningTime="2026-11-04 01:38:06.562787912 +0000 UTC m=+217.278974769"
Nov 04 01:38:06.728637 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:06.728614    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:07.437861 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:07.437829    5039 generic.go:356] "Generic (PLEG): container finished" podID="a5444637-8b12-40d2-b791-689a647e0c90" containerID="10f5ed8bc5eac7cfdcfa20fedfe5e2c414fd2354aeb30789cdc57d56fb4a24a6" exitCode=0
Nov 04 01:38:07.438260 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:07.437909    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97" event={"ID":"a5444637-8b12-40d2-b791-689a647e0c90","Type":"ContainerDied","Data":"10f5ed8bc5eac7cfdcfa20fedfe5e2c414fd2354aeb30789cdc57d56fb4a24a6"}
Nov 04 01:38:07.733021 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:07.732956    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:08.442154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:08.442130    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:08.443145 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:08.443128    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:38:10.451925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:10.451888    5039 generic.go:356] "Generic (PLEG): container finished" podID="a5444637-8b12-40d2-b791-689a647e0c90" containerID="632bae54f15cada746c7387493536b8d8511524b3a358bab5290052bbd7817da" exitCode=0
Nov 04 01:38:10.452281 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:10.451979    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97" event={"ID":"a5444637-8b12-40d2-b791-689a647e0c90","Type":"ContainerDied","Data":"632bae54f15cada746c7387493536b8d8511524b3a358bab5290052bbd7817da"}
Nov 04 01:38:17.484395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:17.484362    5039 generic.go:356] "Generic (PLEG): container finished" podID="a5444637-8b12-40d2-b791-689a647e0c90" containerID="5e33e21e570f1e2bd72c06f8ac72524de7c2b0613db392d5282acb09707bd446" exitCode=0
Nov 04 01:38:17.484732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:17.484447    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97" event={"ID":"a5444637-8b12-40d2-b791-689a647e0c90","Type":"ContainerDied","Data":"5e33e21e570f1e2bd72c06f8ac72524de7c2b0613db392d5282acb09707bd446"}
Nov 04 01:38:18.612816 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.612794    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:38:18.763767 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.763714    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td\" (UniqueName: \"kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td\") pod \"a5444637-8b12-40d2-b791-689a647e0c90\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") "
Nov 04 01:38:18.763877 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.763811    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util\") pod \"a5444637-8b12-40d2-b791-689a647e0c90\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") "
Nov 04 01:38:18.763933 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.763882    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle\") pod \"a5444637-8b12-40d2-b791-689a647e0c90\" (UID: \"a5444637-8b12-40d2-b791-689a647e0c90\") "
Nov 04 01:38:18.764417 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.764391    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle" pod "a5444637-8b12-40d2-b791-689a647e0c90" (UID: "a5444637-8b12-40d2-b791-689a647e0c90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:18.765964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.765942    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td" pod "a5444637-8b12-40d2-b791-689a647e0c90" (UID: "a5444637-8b12-40d2-b791-689a647e0c90"). InnerVolumeSpecName "kube-api-access-8b6td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:18.770016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.769989    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util" pod "a5444637-8b12-40d2-b791-689a647e0c90" (UID: "a5444637-8b12-40d2-b791-689a647e0c90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:18.865348 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.865326    5039 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-util\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:18.865348 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.865346    5039 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5444637-8b12-40d2-b791-689a647e0c90-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:18.865474 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:18.865355    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b6td\" (UniqueName: \"kubernetes.io/projected/a5444637-8b12-40d2-b791-689a647e0c90-kube-api-access-8b6td\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:19.496344 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:19.496314    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97" event={"ID":"a5444637-8b12-40d2-b791-689a647e0c90","Type":"ContainerDied","Data":"9851e833875265b9fbf3e9c93bd1ab7280d5aa84d290d3c4353152d87832dd7d"}
Nov 04 01:38:19.496482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:19.496349    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdlk97"
Nov 04 01:38:19.496482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:19.496358    5039 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9851e833875265b9fbf3e9c93bd1ab7280d5aa84d290d3c4353152d87832dd7d"
Nov 04 01:38:20.775480 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:20.775446    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:22.750923 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:38:22.750889    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:38:22.813848 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.813812    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"]
Nov 04 01:38:22.814141 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814128    5039 cpu_manager.go:414] "containerMap: removing container" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="extract"
Nov 04 01:38:22.814236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814143    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="extract"
Nov 04 01:38:22.814236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814156    5039 cpu_manager.go:414] "containerMap: removing container" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="util"
Nov 04 01:38:22.814236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814161    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="util"
Nov 04 01:38:22.814236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814174    5039 cpu_manager.go:414] "containerMap: removing container" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="pull"
Nov 04 01:38:22.814236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814180    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="pull"
Nov 04 01:38:22.814236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.814238    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5444637-8b12-40d2-b791-689a647e0c90" containerName="extract"
Nov 04 01:38:22.879552 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.879528    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"]
Nov 04 01:38:22.879677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.879655    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:22.932646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.932590    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 04 01:38:22.932646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.932609    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:38:22.933130 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.933112    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-krbpz\""
Nov 04 01:38:22.996527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.996503    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b657774-fcb2-4536-a1a8-19c6a7bd9ae5-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6vffv\" (UID: \"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:22.996648 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:22.996562    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w458\" (UniqueName: \"kubernetes.io/projected/4b657774-fcb2-4536-a1a8-19c6a7bd9ae5-kube-api-access-4w458\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6vffv\" (UID: \"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:23.097693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.097669    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b657774-fcb2-4536-a1a8-19c6a7bd9ae5-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6vffv\" (UID: \"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:23.097807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.097729    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w458\" (UniqueName: \"kubernetes.io/projected/4b657774-fcb2-4536-a1a8-19c6a7bd9ae5-kube-api-access-4w458\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6vffv\" (UID: \"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:23.098002 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.097986    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b657774-fcb2-4536-a1a8-19c6a7bd9ae5-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6vffv\" (UID: \"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:23.157959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.157931    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w458\" (UniqueName: \"kubernetes.io/projected/4b657774-fcb2-4536-a1a8-19c6a7bd9ae5-kube-api-access-4w458\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-6vffv\" (UID: \"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:23.210892 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.210873    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"
Nov 04 01:38:23.489047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.488979    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:23.700653 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:23.700628    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv"]
Nov 04 01:38:23.738789 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:38:23.738764    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b657774_fcb2_4536_a1a8_19c6a7bd9ae5.slice/crio-651b50df3f8e8eb6775e245d75d1dfd4608e0b26b8f58f55986ab1626a6b00f6 WatchSource:0}: Error finding container 651b50df3f8e8eb6775e245d75d1dfd4608e0b26b8f58f55986ab1626a6b00f6: Status 404 returned error can't find the container with id 651b50df3f8e8eb6775e245d75d1dfd4608e0b26b8f58f55986ab1626a6b00f6
Nov 04 01:38:24.516242 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:24.516209    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv" event={"ID":"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5","Type":"ContainerStarted","Data":"651b50df3f8e8eb6775e245d75d1dfd4608e0b26b8f58f55986ab1626a6b00f6"}
Nov 04 01:38:24.734481 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:24.734451    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:25.005003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:25.004978    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:26.771684 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:26.771654    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:27.025714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.025650    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:27.530628 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.530596    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv" event={"ID":"4b657774-fcb2-4536-a1a8-19c6a7bd9ae5","Type":"ContainerStarted","Data":"6d95ccf2a0e9d95819c80ec65dfdb922a04e45fa5ecdfb6b7d85a5e67082e95c"}
Nov 04 01:38:27.595814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.595790    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:27.701004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.700976    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"]
Nov 04 01:38:27.724134 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.724106    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.732976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.732940    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-6vffv" podStartSLOduration=2.529154851 podStartE2EDuration="5.732930395s" totalImagesPullingTime="3.203775544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:27.713857141 +0000 UTC m=+238.430043996" watchObservedRunningTime="2026-11-04 01:38:27.732930395 +0000 UTC m=+238.449117250"
Nov 04 01:38:27.778450 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.778423    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 01:38:27.778992 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.778969    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 01:38:27.780601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.780561    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 01:38:27.780727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.780672    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-xh4nj\""
Nov 04 01:38:27.780784 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.780734    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 01:38:27.780831 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.780783    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 01:38:27.837672 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.837644    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df782e9e-bd95-4a78-8e52-95645dc14b99-manifests\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.837772 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.837682    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c589g\" (UniqueName: \"kubernetes.io/projected/df782e9e-bd95-4a78-8e52-95645dc14b99-kube-api-access-c589g\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.837836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.837818    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-metrics-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.837888 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.837873    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df782e9e-bd95-4a78-8e52-95645dc14b99-tmp\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.837941 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.837908    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-webhook-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.838033 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.838013    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939229 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939201    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-webhook-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939233    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939316    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df782e9e-bd95-4a78-8e52-95645dc14b99-manifests\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939347    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c589g\" (UniqueName: \"kubernetes.io/projected/df782e9e-bd95-4a78-8e52-95645dc14b99-kube-api-access-c589g\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939517 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939498    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-metrics-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939604 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939589    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df782e9e-bd95-4a78-8e52-95645dc14b99-tmp\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.939772 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.939746    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df782e9e-bd95-4a78-8e52-95645dc14b99-manifests\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.941799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.941773    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df782e9e-bd95-4a78-8e52-95645dc14b99-tmp\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.941872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.941808    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-webhook-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.941872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.941844    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.941951 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.941920    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/df782e9e-bd95-4a78-8e52-95645dc14b99-metrics-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:27.976075 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:27.976055    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"]
Nov 04 01:38:28.007192 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.007162    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c589g\" (UniqueName: \"kubernetes.io/projected/df782e9e-bd95-4a78-8e52-95645dc14b99-kube-api-access-c589g\") pod \"opendatahub-operator-controller-manager-7565df5f-vwrdz\" (UID: \"df782e9e-bd95-4a78-8e52-95645dc14b99\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:28.008180 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.008101    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:28.078319 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.078258    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:28.297287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.297250    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:28.585222 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.585190    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:28.601614 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.601595    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"]
Nov 04 01:38:28.604156 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:38:28.604119    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf782e9e_bd95_4a78_8e52_95645dc14b99.slice/crio-c0d0149751fc1d38e158ee1ae91531813e490bed4229cfa6dc86d74f3cf05bbf WatchSource:0}: Error finding container c0d0149751fc1d38e158ee1ae91531813e490bed4229cfa6dc86d74f3cf05bbf: Status 404 returned error can't find the container with id c0d0149751fc1d38e158ee1ae91531813e490bed4229cfa6dc86d74f3cf05bbf
Nov 04 01:38:28.970446 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:28.970415    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:29.237538 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:29.237466    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:29.541120 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:29.541056    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz" event={"ID":"df782e9e-bd95-4a78-8e52-95645dc14b99","Type":"ContainerStarted","Data":"c0d0149751fc1d38e158ee1ae91531813e490bed4229cfa6dc86d74f3cf05bbf"}
Nov 04 01:38:30.891732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:30.891700    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:31.347308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:31.347270    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:31.786182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:31.786090    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:32.821144 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:32.821092    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:33.079516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:33.079464    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:33.448811 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:33.448755    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:33.559971 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:33.559945    5039 generic.go:356] "Generic (PLEG): container finished" podID="df782e9e-bd95-4a78-8e52-95645dc14b99" containerID="bf5df81739555148a7c50e5dabbcc98820f5a09fe3007aeb2c7308808e53e237" exitCode=0
Nov 04 01:38:33.560070 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:33.560023    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz" event={"ID":"df782e9e-bd95-4a78-8e52-95645dc14b99","Type":"ContainerDied","Data":"bf5df81739555148a7c50e5dabbcc98820f5a09fe3007aeb2c7308808e53e237"}
Nov 04 01:38:33.701108 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:33.701046    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:34.102930 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:34.102900    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:34.567032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:34.566998    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz" event={"ID":"df782e9e-bd95-4a78-8e52-95645dc14b99","Type":"ContainerStarted","Data":"10bbd992516303664d057e7fe5dff6f596cf4ba2f01b4c46d979a3433115caf1"}
Nov 04 01:38:34.567201 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:34.567118    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:34.703893 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:34.703799    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz" podStartSLOduration=2.054362389 podStartE2EDuration="7.703786079s" totalImagesPullingTime="5.195871279s" totalInitContainerRuntime="453.552411ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:34.702226104 +0000 UTC m=+245.418412959" watchObservedRunningTime="2026-11-04 01:38:34.703786079 +0000 UTC m=+245.419972931"
Nov 04 01:38:34.937596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:34.937556    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:35.216585 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:35.216535    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:37.632686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.632657    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"]
Nov 04 01:38:37.634791 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.634775    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:37.714411 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.714391    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msklk\" (UniqueName: \"kubernetes.io/projected/5049d2db-2b1f-4df4-b779-db8213773d63-kube-api-access-msklk\") pod \"dns-operator-controller-manager-8557cbdb84-kj776\" (UID: \"5049d2db-2b1f-4df4-b779-db8213773d63\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:37.736058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.736029    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lf9n9\""
Nov 04 01:38:37.736058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.736037    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 01:38:37.736196 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.736075    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 04 01:38:37.736196 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.736095    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 01:38:37.739042 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.739023    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"]
Nov 04 01:38:37.815026 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.814999    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msklk\" (UniqueName: \"kubernetes.io/projected/5049d2db-2b1f-4df4-b779-db8213773d63-kube-api-access-msklk\") pod \"dns-operator-controller-manager-8557cbdb84-kj776\" (UID: \"5049d2db-2b1f-4df4-b779-db8213773d63\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:37.820823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.820799    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:37.877959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.877933    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msklk\" (UniqueName: \"kubernetes.io/projected/5049d2db-2b1f-4df4-b779-db8213773d63-kube-api-access-msklk\") pod \"dns-operator-controller-manager-8557cbdb84-kj776\" (UID: \"5049d2db-2b1f-4df4-b779-db8213773d63\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:37.945085 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:37.945021    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:38.094978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:38.094942    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:38.317650 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:38.317627    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"]
Nov 04 01:38:38.319594 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:38:38.319548    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5049d2db_2b1f_4df4_b779_db8213773d63.slice/crio-aac39db0beb6765af2c264ab87908304131a167a52d356b8135fa4393eaa485b WatchSource:0}: Error finding container aac39db0beb6765af2c264ab87908304131a167a52d356b8135fa4393eaa485b: Status 404 returned error can't find the container with id aac39db0beb6765af2c264ab87908304131a167a52d356b8135fa4393eaa485b
Nov 04 01:38:38.503734 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:38.503697    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:38.583725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:38.583656    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776" event={"ID":"5049d2db-2b1f-4df4-b779-db8213773d63","Type":"ContainerStarted","Data":"aac39db0beb6765af2c264ab87908304131a167a52d356b8135fa4393eaa485b"}
Nov 04 01:38:38.752781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:38.752755    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:39.346339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.346300    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:39.562541 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.562506    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-8bf4k"]
Nov 04 01:38:39.572757 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.572737    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.615029 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.614967    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:39.628261 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.628238    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8bf4k"]
Nov 04 01:38:39.675823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.675801    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 01:38:39.675823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.675807    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-l54wb\""
Nov 04 01:38:39.678003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.677986    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 01:38:39.730990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.730965    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eecdf4c-1a40-4028-b148-7b5f234234a7-bound-sa-token\") pod \"cert-manager-597b95cb84-8bf4k\" (UID: \"8eecdf4c-1a40-4028-b148-7b5f234234a7\") " pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.731103 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.730993    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxvnp\" (UniqueName: \"kubernetes.io/projected/8eecdf4c-1a40-4028-b148-7b5f234234a7-kube-api-access-gxvnp\") pod \"cert-manager-597b95cb84-8bf4k\" (UID: \"8eecdf4c-1a40-4028-b148-7b5f234234a7\") " pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.832060 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.832032    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eecdf4c-1a40-4028-b148-7b5f234234a7-bound-sa-token\") pod \"cert-manager-597b95cb84-8bf4k\" (UID: \"8eecdf4c-1a40-4028-b148-7b5f234234a7\") " pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.832389 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.832068    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxvnp\" (UniqueName: \"kubernetes.io/projected/8eecdf4c-1a40-4028-b148-7b5f234234a7-kube-api-access-gxvnp\") pod \"cert-manager-597b95cb84-8bf4k\" (UID: \"8eecdf4c-1a40-4028-b148-7b5f234234a7\") " pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.868665 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.868604    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:39.893003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.892975    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxvnp\" (UniqueName: \"kubernetes.io/projected/8eecdf4c-1a40-4028-b148-7b5f234234a7-kube-api-access-gxvnp\") pod \"cert-manager-597b95cb84-8bf4k\" (UID: \"8eecdf4c-1a40-4028-b148-7b5f234234a7\") " pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.893117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.893100    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8eecdf4c-1a40-4028-b148-7b5f234234a7-bound-sa-token\") pod \"cert-manager-597b95cb84-8bf4k\" (UID: \"8eecdf4c-1a40-4028-b148-7b5f234234a7\") " pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:39.893929 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:39.893911    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8bf4k"
Nov 04 01:38:40.272169 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:40.272083    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8bf4k"]
Nov 04 01:38:40.594005 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:40.593971    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8bf4k" event={"ID":"8eecdf4c-1a40-4028-b148-7b5f234234a7","Type":"ContainerStarted","Data":"d538d958439b16171f9fc56e5ba6d4209285725f484135cb5213080442fc4c70"}
Nov 04 01:38:42.605827 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:42.605793    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776" event={"ID":"5049d2db-2b1f-4df4-b779-db8213773d63","Type":"ContainerStarted","Data":"e6a7d5c92d579812050ffb826948a509ae15630007ad8a71cd0fd97785ea11ae"}
Nov 04 01:38:42.606538 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:42.606517    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:42.724739 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:42.724680    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776" podStartSLOduration=2.251084732 podStartE2EDuration="5.724666299s" totalImagesPullingTime="3.473581567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:42.723207901 +0000 UTC m=+253.439394756" watchObservedRunningTime="2026-11-04 01:38:42.724666299 +0000 UTC m=+253.440853154"
Nov 04 01:38:43.009742 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.009660    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"]
Nov 04 01:38:43.011896 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.011874    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:38:43.063989 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.063961    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bb9d8\""
Nov 04 01:38:43.076301 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.076275    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"]
Nov 04 01:38:43.160964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.160928    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d54gm\" (UniqueName: \"kubernetes.io/projected/4a6dbbf5-313a-4ed8-afb6-6e1a94f86033-kube-api-access-d54gm\") pod \"authorino-operator-7b7bb4bc77-9x6jf\" (UID: \"4a6dbbf5-313a-4ed8-afb6-6e1a94f86033\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:38:43.261965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.261888    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d54gm\" (UniqueName: \"kubernetes.io/projected/4a6dbbf5-313a-4ed8-afb6-6e1a94f86033-kube-api-access-d54gm\") pod \"authorino-operator-7b7bb4bc77-9x6jf\" (UID: \"4a6dbbf5-313a-4ed8-afb6-6e1a94f86033\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:38:43.325965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.325933    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d54gm\" (UniqueName: \"kubernetes.io/projected/4a6dbbf5-313a-4ed8-afb6-6e1a94f86033-kube-api-access-d54gm\") pod \"authorino-operator-7b7bb4bc77-9x6jf\" (UID: \"4a6dbbf5-313a-4ed8-afb6-6e1a94f86033\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:38:43.611821 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.611785    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8bf4k" event={"ID":"8eecdf4c-1a40-4028-b148-7b5f234234a7","Type":"ContainerStarted","Data":"b10c48993dbb4e2efb7855ca1235418014edc047b7f138170a7a9b606285c861"}
Nov 04 01:38:43.623808 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.623788    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:38:43.702540 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.702513    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:43.729047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.728992    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-8bf4k" podStartSLOduration=1.552460619 podStartE2EDuration="4.728975484s" totalImagesPullingTime="3.176514865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:43.728775297 +0000 UTC m=+254.444962152" watchObservedRunningTime="2026-11-04 01:38:43.728975484 +0000 UTC m=+254.445162341"
Nov 04 01:38:43.999824 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:43.999796    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"]
Nov 04 01:38:44.001806 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:38:44.001775    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a6dbbf5_313a_4ed8_afb6_6e1a94f86033.slice/crio-d7fe6e166c2fcf422252394addfdb8be04e7376629730b0bcea216c91d487fa5 WatchSource:0}: Error finding container d7fe6e166c2fcf422252394addfdb8be04e7376629730b0bcea216c91d487fa5: Status 404 returned error can't find the container with id d7fe6e166c2fcf422252394addfdb8be04e7376629730b0bcea216c91d487fa5
Nov 04 01:38:44.617123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:44.617085    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf" event={"ID":"4a6dbbf5-313a-4ed8-afb6-6e1a94f86033","Type":"ContainerStarted","Data":"d7fe6e166c2fcf422252394addfdb8be04e7376629730b0bcea216c91d487fa5"}
Nov 04 01:38:44.737912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:44.737878    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:45.572984 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:45.572956    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz"
Nov 04 01:38:45.701434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:45.701396    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:46.626947 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:46.626918    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf" event={"ID":"4a6dbbf5-313a-4ed8-afb6-6e1a94f86033","Type":"ContainerStarted","Data":"1f4199b73b1860f544729c87ec891e47a7bc7042c957cb5ced71ed72f89b884c"}
Nov 04 01:38:46.627102 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:46.627087    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:38:46.745241 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:46.745196    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf" podStartSLOduration=2.221224737 podStartE2EDuration="4.745183785s" totalImagesPullingTime="2.523959048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:46.742664689 +0000 UTC m=+257.458851544" watchObservedRunningTime="2026-11-04 01:38:46.745183785 +0000 UTC m=+257.461370640"
Nov 04 01:38:47.988892 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:47.988863    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:50.570474 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.570411    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:50.572107 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.572085    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"]
Nov 04 01:38:50.574059 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.574045    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.589076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.589056    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"]
Nov 04 01:38:50.621686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.621651    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14205fbd-03a1-442b-9907-4364028e4a24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-cjjc7\" (UID: \"14205fbd-03a1-442b-9907-4364028e4a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.621796 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.621715    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx44x\" (UniqueName: \"kubernetes.io/projected/14205fbd-03a1-442b-9907-4364028e4a24-kube-api-access-vx44x\") pod \"kuadrant-operator-controller-manager-678d87766f-cjjc7\" (UID: \"14205fbd-03a1-442b-9907-4364028e4a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.652434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.652406    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xk67l\""
Nov 04 01:38:50.723041 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.723012    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14205fbd-03a1-442b-9907-4364028e4a24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-cjjc7\" (UID: \"14205fbd-03a1-442b-9907-4364028e4a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.723141 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.723059    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx44x\" (UniqueName: \"kubernetes.io/projected/14205fbd-03a1-442b-9907-4364028e4a24-kube-api-access-vx44x\") pod \"kuadrant-operator-controller-manager-678d87766f-cjjc7\" (UID: \"14205fbd-03a1-442b-9907-4364028e4a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.723380 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.723364    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14205fbd-03a1-442b-9907-4364028e4a24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-cjjc7\" (UID: \"14205fbd-03a1-442b-9907-4364028e4a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.783722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.783702    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx44x\" (UniqueName: \"kubernetes.io/projected/14205fbd-03a1-442b-9907-4364028e4a24-kube-api-access-vx44x\") pod \"kuadrant-operator-controller-manager-678d87766f-cjjc7\" (UID: \"14205fbd-03a1-442b-9907-4364028e4a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.884747 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.884730    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:50.934912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:50.934876    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:51.362286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:51.362256    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"]
Nov 04 01:38:51.365590 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:38:51.365548    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14205fbd_03a1_442b_9907_4364028e4a24.slice/crio-1e4eeecc961334d2afe1a952db82c4ee0704671b9d97a3aaab22e213ae4951b2 WatchSource:0}: Error finding container 1e4eeecc961334d2afe1a952db82c4ee0704671b9d97a3aaab22e213ae4951b2: Status 404 returned error can't find the container with id 1e4eeecc961334d2afe1a952db82c4ee0704671b9d97a3aaab22e213ae4951b2
Nov 04 01:38:51.510032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:51.510005    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:51.648650 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:51.648585    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7" event={"ID":"14205fbd-03a1-442b-9907-4364028e4a24","Type":"ContainerStarted","Data":"1e4eeecc961334d2afe1a952db82c4ee0704671b9d97a3aaab22e213ae4951b2"}
Nov 04 01:38:52.750668 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:38:52.750636    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:38:54.619662 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:54.619633    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776"
Nov 04 01:38:54.943217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:54.943146    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:55.201131 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:55.201039    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:56.672835 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:56.672800    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7" event={"ID":"14205fbd-03a1-442b-9907-4364028e4a24","Type":"ContainerStarted","Data":"134f9b47e254284b219b7399bb1a3cd47183e9165f6c42cbe3d757e94426f754"}
Nov 04 01:38:56.673215 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:56.672893    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:38:56.890374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:56.890325    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7" podStartSLOduration=1.975179061 podStartE2EDuration="6.890312113s" totalImagesPullingTime="4.915133052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:56.889079438 +0000 UTC m=+267.605266297" watchObservedRunningTime="2026-11-04 01:38:56.890312113 +0000 UTC m=+267.606498968"
Nov 04 01:38:57.632512 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:38:57.632486    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf"
Nov 04 01:39:00.104737 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:00.104704    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:00.349624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:00.349590    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:00.604491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:00.604462    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:00.849086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:00.849058    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:01.097407 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:01.097380    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:03.579999 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:03.579961    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:05.530590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:05.530540    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:05.834939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:05.834845    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:06.265166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:06.265134    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:07.453581 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:07.453547    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:07.679066 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:07.679037    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7"
Nov 04 01:39:07.713908 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:07.713837    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:08.016130 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:08.016043    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:08.264623 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:08.264549    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:08.561482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:08.561451    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:08.852959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:08.852928    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:09.260162 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:09.260075    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:09.547088 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:09.547016    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:10.686100 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.686066    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:10.689624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.689601    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:10.706560 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.706527    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:10.741478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.741454    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 01:39:10.785598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.785548    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file\") pod \"limitador-limitador-74dfcfb894-jk4fq\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:10.785725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.785663    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh9rs\" (UniqueName: \"kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs\") pod \"limitador-limitador-74dfcfb894-jk4fq\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:10.790921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.790899    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5hchf\""
Nov 04 01:39:10.792386 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.792363    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:10.886789 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.886760    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file\") pod \"limitador-limitador-74dfcfb894-jk4fq\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:10.886960 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.886800    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh9rs\" (UniqueName: \"kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs\") pod \"limitador-limitador-74dfcfb894-jk4fq\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:10.887348 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.887318    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file\") pod \"limitador-limitador-74dfcfb894-jk4fq\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:10.924297 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.924270    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:10.948702 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:10.948643    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh9rs\" (UniqueName: \"kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs\") pod \"limitador-limitador-74dfcfb894-jk4fq\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:11.001909 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:11.001880    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:11.043695 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:11.043669    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:11.324033 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:11.323938    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:11.375170 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:11.375148    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:11.377607 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:11.377562    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e778d6e_215a_4c56_9b04_3c8faad18c54.slice/crio-dad28e76b96b5cb8fcb501eb186e4c5dc7fb6d5645413b2c9cbd7cb9c3d09539 WatchSource:0}: Error finding container dad28e76b96b5cb8fcb501eb186e4c5dc7fb6d5645413b2c9cbd7cb9c3d09539: Status 404 returned error can't find the container with id dad28e76b96b5cb8fcb501eb186e4c5dc7fb6d5645413b2c9cbd7cb9c3d09539
Nov 04 01:39:11.740827 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:11.740799    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq" event={"ID":"5e778d6e-215a-4c56-9b04-3c8faad18c54","Type":"ContainerStarted","Data":"dad28e76b96b5cb8fcb501eb186e4c5dc7fb6d5645413b2c9cbd7cb9c3d09539"}
Nov 04 01:39:12.187240 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.187210    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jh68b"]
Nov 04 01:39:12.191686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.191664    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:12.246356 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.246331    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n2dfd\""
Nov 04 01:39:12.248212 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.248186    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jh68b"]
Nov 04 01:39:12.299147 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.299118    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4w4q\" (UniqueName: \"kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q\") pod \"authorino-57f7f8659f-jh68b\" (UID: \"4fc479df-79e5-4a5e-913e-2f39e9844720\") " pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:12.400488 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.400464    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4w4q\" (UniqueName: \"kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q\") pod \"authorino-57f7f8659f-jh68b\" (UID: \"4fc479df-79e5-4a5e-913e-2f39e9844720\") " pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:12.443300 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.443203    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:12.462752 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.462725    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4w4q\" (UniqueName: \"kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q\") pod \"authorino-57f7f8659f-jh68b\" (UID: \"4fc479df-79e5-4a5e-913e-2f39e9844720\") " pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:12.503011 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.502989    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:12.702242 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.702154    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:12.875996 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:12.875804    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jh68b"]
Nov 04 01:39:12.878711 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:12.878681    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fc479df_79e5_4a5e_913e_2f39e9844720.slice/crio-dfdf077239ab84c2b9fc4259f1a7c41c84e53d0eca672b5deff863bb5df5a0aa WatchSource:0}: Error finding container dfdf077239ab84c2b9fc4259f1a7c41c84e53d0eca672b5deff863bb5df5a0aa: Status 404 returned error can't find the container with id dfdf077239ab84c2b9fc4259f1a7c41c84e53d0eca672b5deff863bb5df5a0aa
Nov 04 01:39:13.752228 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:13.752185    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jh68b" event={"ID":"4fc479df-79e5-4a5e-913e-2f39e9844720","Type":"ContainerStarted","Data":"dfdf077239ab84c2b9fc4259f1a7c41c84e53d0eca672b5deff863bb5df5a0aa"}
Nov 04 01:39:13.843256 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:13.843226    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bbc54"]
Nov 04 01:39:13.863986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:13.863961    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:39:13.903498 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:13.903477    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bbc54"]
Nov 04 01:39:13.914796 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:13.914768    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhxg\" (UniqueName: \"kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg\") pod \"authorino-56c6c9ddd5-bbc54\" (UID: \"f4d3ee81-fb03-4e8a-adc5-9732d495e675\") " pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:39:14.015780 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.015722    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhxg\" (UniqueName: \"kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg\") pod \"authorino-56c6c9ddd5-bbc54\" (UID: \"f4d3ee81-fb03-4e8a-adc5-9732d495e675\") " pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:39:14.076819 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.076786    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhxg\" (UniqueName: \"kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg\") pod \"authorino-56c6c9ddd5-bbc54\" (UID: \"f4d3ee81-fb03-4e8a-adc5-9732d495e675\") " pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:39:14.188331 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.188305    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:39:14.645190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.645154    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bbc54"]
Nov 04 01:39:14.647695 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:14.647666    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4d3ee81_fb03_4e8a_adc5_9732d495e675.slice/crio-cd53436f7f6f7aa6bb2f90030e4942434f93fca5447b058e0beb9e0b264138da WatchSource:0}: Error finding container cd53436f7f6f7aa6bb2f90030e4942434f93fca5447b058e0beb9e0b264138da: Status 404 returned error can't find the container with id cd53436f7f6f7aa6bb2f90030e4942434f93fca5447b058e0beb9e0b264138da
Nov 04 01:39:14.758837 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.758812    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq" event={"ID":"5e778d6e-215a-4c56-9b04-3c8faad18c54","Type":"ContainerStarted","Data":"5952b78d8739ebb0b63ae6e7dc30c77dc9219e2064b4fd8e17a074b8ca75cc26"}
Nov 04 01:39:14.758980 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.758892    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:14.760177 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.760150    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54" event={"ID":"f4d3ee81-fb03-4e8a-adc5-9732d495e675","Type":"ContainerStarted","Data":"cd53436f7f6f7aa6bb2f90030e4942434f93fca5447b058e0beb9e0b264138da"}
Nov 04 01:39:14.873805 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:14.873754    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq" podStartSLOduration=1.9684233180000001 podStartE2EDuration="4.873740883s" totalImagesPullingTime="2.905317565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:14.873236341 +0000 UTC m=+285.589423196" watchObservedRunningTime="2026-11-04 01:39:14.873740883 +0000 UTC m=+285.589927739"
Nov 04 01:39:15.130507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:15.130478    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:15.382429 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:15.382353    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:15.807273 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:15.807202    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:17.777466 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:17.777437    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jh68b" event={"ID":"4fc479df-79e5-4a5e-913e-2f39e9844720","Type":"ContainerStarted","Data":"822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa"}
Nov 04 01:39:17.779083 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:17.779055    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54" event={"ID":"f4d3ee81-fb03-4e8a-adc5-9732d495e675","Type":"ContainerStarted","Data":"22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06"}
Nov 04 01:39:17.891754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:17.891538    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-jh68b" podStartSLOduration=2.053392113 podStartE2EDuration="5.891521665s" totalImagesPullingTime="3.838129552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:17.890777098 +0000 UTC m=+288.606963955" watchObservedRunningTime="2026-11-04 01:39:17.891521665 +0000 UTC m=+288.607708519"
Nov 04 01:39:18.003606 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:18.003539    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54" podStartSLOduration=2.755057331 podStartE2EDuration="5.003524562s" totalImagesPullingTime="2.248467231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:18.003465116 +0000 UTC m=+288.719651972" watchObservedRunningTime="2026-11-04 01:39:18.003524562 +0000 UTC m=+288.719711414"
Nov 04 01:39:18.030123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:18.030043    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jh68b"]
Nov 04 01:39:19.764646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:19.764607    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:19.788297 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:19.788268    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-jh68b" podUID="4fc479df-79e5-4a5e-913e-2f39e9844720" containerName="authorino" containerID="cri-o://822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa" gracePeriod=30
Nov 04 01:39:20.019990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.019934    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:20.029666 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.029645    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:20.168750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.168718    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q\" (UniqueName: \"kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q\") pod \"4fc479df-79e5-4a5e-913e-2f39e9844720\" (UID: \"4fc479df-79e5-4a5e-913e-2f39e9844720\") "
Nov 04 01:39:20.170940 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.170914    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q" pod "4fc479df-79e5-4a5e-913e-2f39e9844720" (UID: "4fc479df-79e5-4a5e-913e-2f39e9844720"). InnerVolumeSpecName "kube-api-access-v4w4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:39:20.270096 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.270024    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4w4q\" (UniqueName: \"kubernetes.io/projected/4fc479df-79e5-4a5e-913e-2f39e9844720-kube-api-access-v4w4q\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:39:20.793954 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.793922    5039 generic.go:356] "Generic (PLEG): container finished" podID="4fc479df-79e5-4a5e-913e-2f39e9844720" containerID="822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa" exitCode=0
Nov 04 01:39:20.794342 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.793961    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jh68b"
Nov 04 01:39:20.794342 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.794003    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jh68b" event={"ID":"4fc479df-79e5-4a5e-913e-2f39e9844720","Type":"ContainerDied","Data":"822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa"}
Nov 04 01:39:20.794342 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.794044    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jh68b" event={"ID":"4fc479df-79e5-4a5e-913e-2f39e9844720","Type":"ContainerDied","Data":"dfdf077239ab84c2b9fc4259f1a7c41c84e53d0eca672b5deff863bb5df5a0aa"}
Nov 04 01:39:20.794342 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.794060    5039 scope.go:122] "RemoveContainer" containerID="822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa"
Nov 04 01:39:20.804320 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.804304    5039 scope.go:122] "RemoveContainer" containerID="822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa"
Nov 04 01:39:20.804620 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:39:20.804595    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa\": container with ID starting with 822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa not found: ID does not exist" containerID="822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa"
Nov 04 01:39:20.804704 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.804628    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa"} err="failed to get container status \"822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa\": rpc error: code = NotFound desc = could not find container \"822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa\": container with ID starting with 822a5903af5a3f74af63456795a8668cc58cbd4a8751c6250249a708e62717aa not found: ID does not exist"
Nov 04 01:39:20.955670 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.955647    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jh68b"]
Nov 04 01:39:20.959551 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:20.959532    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jh68b"]
Nov 04 01:39:21.133715 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:21.133689    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:21.408579 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:21.408504    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fc479df-79e5-4a5e-913e-2f39e9844720" path="/var/lib/kubelet/pods/4fc479df-79e5-4a5e-913e-2f39e9844720/volumes"
Nov 04 01:39:22.750605 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:39:22.750554    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:39:24.551024 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:24.550988    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:24.797001 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:24.796969    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:25.767643 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:25.767620    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:28.316688 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:28.316652    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:28.317127 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:28.316862    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq" podUID="5e778d6e-215a-4c56-9b04-3c8faad18c54" containerName="limitador" containerID="cri-o://5952b78d8739ebb0b63ae6e7dc30c77dc9219e2064b4fd8e17a074b8ca75cc26" gracePeriod=30
Nov 04 01:39:28.833273 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:28.833245    5039 generic.go:356] "Generic (PLEG): container finished" podID="5e778d6e-215a-4c56-9b04-3c8faad18c54" containerID="5952b78d8739ebb0b63ae6e7dc30c77dc9219e2064b4fd8e17a074b8ca75cc26" exitCode=0
Nov 04 01:39:28.833406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:28.833283    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq" event={"ID":"5e778d6e-215a-4c56-9b04-3c8faad18c54","Type":"ContainerDied","Data":"5952b78d8739ebb0b63ae6e7dc30c77dc9219e2064b4fd8e17a074b8ca75cc26"}
Nov 04 01:39:28.860512 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:28.860492    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:29.034666 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.034600    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file\" (UniqueName: \"kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file\") pod \"5e778d6e-215a-4c56-9b04-3c8faad18c54\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") "
Nov 04 01:39:29.034666 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.034641    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs\" (UniqueName: \"kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs\") pod \"5e778d6e-215a-4c56-9b04-3c8faad18c54\" (UID: \"5e778d6e-215a-4c56-9b04-3c8faad18c54\") "
Nov 04 01:39:29.034950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.034930    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file" pod "5e778d6e-215a-4c56-9b04-3c8faad18c54" (UID: "5e778d6e-215a-4c56-9b04-3c8faad18c54"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:39:29.036738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.036718    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs" pod "5e778d6e-215a-4c56-9b04-3c8faad18c54" (UID: "5e778d6e-215a-4c56-9b04-3c8faad18c54"). InnerVolumeSpecName "kube-api-access-hh9rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:39:29.135485 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.135460    5039 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5e778d6e-215a-4c56-9b04-3c8faad18c54-config-file\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:39:29.135485 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.135483    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh9rs\" (UniqueName: \"kubernetes.io/projected/5e778d6e-215a-4c56-9b04-3c8faad18c54-kube-api-access-hh9rs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:39:29.415508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.415480    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:39:29.418209 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.418186    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:39:29.838444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.838374    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq" event={"ID":"5e778d6e-215a-4c56-9b04-3c8faad18c54","Type":"ContainerDied","Data":"dad28e76b96b5cb8fcb501eb186e4c5dc7fb6d5645413b2c9cbd7cb9c3d09539"}
Nov 04 01:39:29.838444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.838424    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"
Nov 04 01:39:29.838671 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:29.838453    5039 scope.go:122] "RemoveContainer" containerID="5952b78d8739ebb0b63ae6e7dc30c77dc9219e2064b4fd8e17a074b8ca75cc26"
Nov 04 01:39:30.000592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:30.000543    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:30.049506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:30.049480    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-jk4fq"]
Nov 04 01:39:31.404857 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:31.404824    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e778d6e-215a-4c56-9b04-3c8faad18c54" path="/var/lib/kubelet/pods/5e778d6e-215a-4c56-9b04-3c8faad18c54/volumes"
Nov 04 01:39:31.633026 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:31.632995    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:31.876616 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:31.876586    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:33.551106 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:33.551059    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:33.800209 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:33.800173    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:34.245629 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.245598    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"]
Nov 04 01:39:34.246007 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.245995    5039 cpu_manager.go:414] "containerMap: removing container" podUID="5e778d6e-215a-4c56-9b04-3c8faad18c54" containerName="limitador"
Nov 04 01:39:34.246050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.246009    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e778d6e-215a-4c56-9b04-3c8faad18c54" containerName="limitador"
Nov 04 01:39:34.246050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.246022    5039 cpu_manager.go:414] "containerMap: removing container" podUID="4fc479df-79e5-4a5e-913e-2f39e9844720" containerName="authorino"
Nov 04 01:39:34.246050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.246027    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fc479df-79e5-4a5e-913e-2f39e9844720" containerName="authorino"
Nov 04 01:39:34.246134 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.246096    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e778d6e-215a-4c56-9b04-3c8faad18c54" containerName="limitador"
Nov 04 01:39:34.246134 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.246105    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fc479df-79e5-4a5e-913e-2f39e9844720" containerName="authorino"
Nov 04 01:39:34.248551 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.248534    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.300724 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.300694    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 04 01:39:34.350294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.350265    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:39:34.350294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.350265    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bjmt5\""
Nov 04 01:39:34.350294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.350269    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:39:34.351739 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.351721    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"]
Nov 04 01:39:34.376927 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.376902    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a08c10ca-6078-4eee-8382-63a36f82533d-data\") pod \"postgres-7cdbf6dc44-b8gg8\" (UID: \"a08c10ca-6078-4eee-8382-63a36f82533d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.377025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.376932    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrmhf\" (UniqueName: \"kubernetes.io/projected/a08c10ca-6078-4eee-8382-63a36f82533d-kube-api-access-rrmhf\") pod \"postgres-7cdbf6dc44-b8gg8\" (UID: \"a08c10ca-6078-4eee-8382-63a36f82533d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.378756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.378718    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:34.478020 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.477997    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a08c10ca-6078-4eee-8382-63a36f82533d-data\") pod \"postgres-7cdbf6dc44-b8gg8\" (UID: \"a08c10ca-6078-4eee-8382-63a36f82533d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.478128 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.478027    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrmhf\" (UniqueName: \"kubernetes.io/projected/a08c10ca-6078-4eee-8382-63a36f82533d-kube-api-access-rrmhf\") pod \"postgres-7cdbf6dc44-b8gg8\" (UID: \"a08c10ca-6078-4eee-8382-63a36f82533d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.478369 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.478349    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a08c10ca-6078-4eee-8382-63a36f82533d-data\") pod \"postgres-7cdbf6dc44-b8gg8\" (UID: \"a08c10ca-6078-4eee-8382-63a36f82533d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.538718 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.538664    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrmhf\" (UniqueName: \"kubernetes.io/projected/a08c10ca-6078-4eee-8382-63a36f82533d-kube-api-access-rrmhf\") pod \"postgres-7cdbf6dc44-b8gg8\" (UID: \"a08c10ca-6078-4eee-8382-63a36f82533d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.559365 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.559346    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:34.608858 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.608783    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"]
Nov 04 01:39:34.612283 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.612247    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.625773 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.625697    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:34.664484 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.664456    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 01:39:34.664628 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.664496    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-94ht7\""
Nov 04 01:39:34.664628 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.664527    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 01:39:34.763946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.763921    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 04 01:39:34.765552 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.765538    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"]
Nov 04 01:39:34.781403 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.781378    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e20f2002-f433-45ac-8e69-8de60bde3443-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.781513 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.781447    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e20f2002-f433-45ac-8e69-8de60bde3443-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.781513 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.781492    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20f2002-f433-45ac-8e69-8de60bde3443-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.781639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.781601    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z8r5\" (UniqueName: \"kubernetes.io/projected/e20f2002-f433-45ac-8e69-8de60bde3443-kube-api-access-7z8r5\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.881954 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.881929    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20f2002-f433-45ac-8e69-8de60bde3443-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.882076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.881972    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z8r5\" (UniqueName: \"kubernetes.io/projected/e20f2002-f433-45ac-8e69-8de60bde3443-kube-api-access-7z8r5\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.882076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.881994    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e20f2002-f433-45ac-8e69-8de60bde3443-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.882076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.882045    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e20f2002-f433-45ac-8e69-8de60bde3443-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.882316 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.882299    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20f2002-f433-45ac-8e69-8de60bde3443-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.882429 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.882398    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e20f2002-f433-45ac-8e69-8de60bde3443-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.883092 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.883075    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e20f2002-f433-45ac-8e69-8de60bde3443-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:34.935315 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.935298    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"]
Nov 04 01:39:34.937305 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:34.937280    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08c10ca_6078_4eee_8382_63a36f82533d.slice/crio-1fe58d61007f383a71fc4e80d2a4ed12fbfc593c2d345fa283a6fb60926c6e79 WatchSource:0}: Error finding container 1fe58d61007f383a71fc4e80d2a4ed12fbfc593c2d345fa283a6fb60926c6e79: Status 404 returned error can't find the container with id 1fe58d61007f383a71fc4e80d2a4ed12fbfc593c2d345fa283a6fb60926c6e79
Nov 04 01:39:34.938417 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.938403    5039 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:39:34.942844 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:34.942819    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z8r5\" (UniqueName: \"kubernetes.io/projected/e20f2002-f433-45ac-8e69-8de60bde3443-kube-api-access-7z8r5\") pod \"kserve-module-controller-manager-7566cf5fc8-wbtv2\" (UID: \"e20f2002-f433-45ac-8e69-8de60bde3443\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:35.228644 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:35.228562    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:35.648408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:35.648376    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"]
Nov 04 01:39:35.651369 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:35.651343    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20f2002_f433_45ac_8e69_8de60bde3443.slice/crio-59b46f02427e60f819abdb7d3e2dc9a63634a876096fe1703c3065843819fdf5 WatchSource:0}: Error finding container 59b46f02427e60f819abdb7d3e2dc9a63634a876096fe1703c3065843819fdf5: Status 404 returned error can't find the container with id 59b46f02427e60f819abdb7d3e2dc9a63634a876096fe1703c3065843819fdf5
Nov 04 01:39:35.866561 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:35.866525    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8" event={"ID":"a08c10ca-6078-4eee-8382-63a36f82533d","Type":"ContainerStarted","Data":"1fe58d61007f383a71fc4e80d2a4ed12fbfc593c2d345fa283a6fb60926c6e79"}
Nov 04 01:39:35.867755 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:35.867729    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2" event={"ID":"e20f2002-f433-45ac-8e69-8de60bde3443","Type":"ContainerStarted","Data":"59b46f02427e60f819abdb7d3e2dc9a63634a876096fe1703c3065843819fdf5"}
Nov 04 01:39:38.886819 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:38.886780    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2" event={"ID":"e20f2002-f433-45ac-8e69-8de60bde3443","Type":"ContainerStarted","Data":"9513ad64d7d43fd06fb06c3ecfe422593a6503019120877884dacfc9ccf8c152"}
Nov 04 01:39:38.887202 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:38.886932    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:39:39.007511 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:39.007458    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2" podStartSLOduration=1.8934003179999999 podStartE2EDuration="5.007438976s" totalImagesPullingTime="3.114038658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:39.006112199 +0000 UTC m=+309.722299056" watchObservedRunningTime="2026-11-04 01:39:39.007438976 +0000 UTC m=+309.723625835"
Nov 04 01:39:39.763437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:39.763339    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:40.008811 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:40.008777    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:40.259595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:40.259533    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:41.904312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:41.904276    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8" event={"ID":"a08c10ca-6078-4eee-8382-63a36f82533d","Type":"ContainerStarted","Data":"0ee7c8dc91cdcfcdcc0cfb907a68144e435cf00df46f0b8b17ae2aeec3fa13b3"}
Nov 04 01:39:41.904780 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:41.904394    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:42.019339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:42.019280    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8" podStartSLOduration=1.382567667 podStartE2EDuration="8.019261342s" totalImagesPullingTime="6.636693675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:42.018682782 +0000 UTC m=+312.734869637" watchObservedRunningTime="2026-11-04 01:39:42.019261342 +0000 UTC m=+312.735448198"
Nov 04 01:39:44.508845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:44.508805    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:44.757711 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:44.757681    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:47.937731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:47.937705    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-b8gg8"
Nov 04 01:39:51.546507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:51.546474    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:51.803037 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:51.802943    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:52.750442 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:39:52.750405    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:39:56.129623 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.129588    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79c79ff55f-qc5qq"]
Nov 04 01:39:56.135146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.135128    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.150983 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.150957    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.151137 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.151119    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qls6h\" (UniqueName: \"kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.151208 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.151169    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.159652 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.159624    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:56.187307 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.187289    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 04 01:39:56.236800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.236775    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 04 01:39:56.236800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.236775    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 04 01:39:56.236945 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.236851    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xblv5\""
Nov 04 01:39:56.238512 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.238494    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c79ff55f-qc5qq"]
Nov 04 01:39:56.252202 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.252182    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.252304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.252233    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.252363 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.252326    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qls6h\" (UniqueName: \"kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.252416 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:39:56.252392    5039 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Nov 04 01:39:56.252470 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:39:56.252459    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert podName:42092908-969c-4e77-9419-941e9d084409 nodeName:}" failed. No retries permitted until 2026-11-04 01:39:56.752438793 +0000 UTC m=+327.468625640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert") pod "maas-controller-79c79ff55f-qc5qq" (UID: "42092908-969c-4e77-9419-941e9d084409") : secret "maas-controller-webhook-cert" not found
Nov 04 01:39:56.254937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.254917    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.313168 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.313147    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qls6h\" (UniqueName: \"kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.408054 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.408000    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:56.652658 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.652628    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:56.725168 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.725105    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8677bb4cb4-5d7s9"]
Nov 04 01:39:56.729209 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.729193    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.755489 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.755468    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-metrics-certs\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.755611 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.755521    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.755611 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.755597    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vks46\" (UniqueName: \"kubernetes.io/projected/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-kube-api-access-vks46\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.755690 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.755659    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-webhook-cert\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.757940 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.757922    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\") pod \"maas-controller-79c79ff55f-qc5qq\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") " pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:56.788617 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.788595    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8677bb4cb4-5d7s9"]
Nov 04 01:39:56.856428 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.856403    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vks46\" (UniqueName: \"kubernetes.io/projected/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-kube-api-access-vks46\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.856541 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.856432    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-webhook-cert\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.856541 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.856502    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-metrics-certs\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.859028 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.858996    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-metrics-certs\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.859120 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.859051    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-webhook-cert\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:56.917051 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:56.917028    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vks46\" (UniqueName: \"kubernetes.io/projected/2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4-kube-api-access-vks46\") pod \"maas-controller-8677bb4cb4-5d7s9\" (UID: \"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4\") " pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:57.042617 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:57.042512    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:39:57.047913 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:57.047892    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:39:57.422432 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:57.422408    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8677bb4cb4-5d7s9"]
Nov 04 01:39:57.424007 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:57.423983    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bbe71a9_9f3e_4700_8fe2_7462e2ed25e4.slice/crio-455852abd290a025ed30b6149cccb66425dab6c5cb6b3029b4c37609489996ab WatchSource:0}: Error finding container 455852abd290a025ed30b6149cccb66425dab6c5cb6b3029b4c37609489996ab: Status 404 returned error can't find the container with id 455852abd290a025ed30b6149cccb66425dab6c5cb6b3029b4c37609489996ab
Nov 04 01:39:57.471434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:57.471413    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c79ff55f-qc5qq"]
Nov 04 01:39:57.473613 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:39:57.473586    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42092908_969c_4e77_9419_941e9d084409.slice/crio-6a08b9f4ff3842be937fb6c8626f676d307c6020cf505cee34bf674a0fcc42e8 WatchSource:0}: Error finding container 6a08b9f4ff3842be937fb6c8626f676d307c6020cf505cee34bf674a0fcc42e8: Status 404 returned error can't find the container with id 6a08b9f4ff3842be937fb6c8626f676d307c6020cf505cee34bf674a0fcc42e8
Nov 04 01:39:57.979071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:57.979038    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c79ff55f-qc5qq" event={"ID":"42092908-969c-4e77-9419-941e9d084409","Type":"ContainerStarted","Data":"6a08b9f4ff3842be937fb6c8626f676d307c6020cf505cee34bf674a0fcc42e8"}
Nov 04 01:39:57.980071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:57.980043    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9" event={"ID":"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4","Type":"ContainerStarted","Data":"455852abd290a025ed30b6149cccb66425dab6c5cb6b3029b4c37609489996ab"}
Nov 04 01:39:59.893607 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:39:59.893557    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2"
Nov 04 01:40:03.623108 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:03.623077    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:03.929669 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:03.929560    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:04.012974 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:04.012943    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c79ff55f-qc5qq" event={"ID":"42092908-969c-4e77-9419-941e9d084409","Type":"ContainerStarted","Data":"3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f"}
Nov 04 01:40:04.013126 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:04.013068    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:40:04.014624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:04.014595    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9" event={"ID":"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4","Type":"ContainerStarted","Data":"32624751eedd57b3073dfc05cd5a289684b891838f5e167bbf23feb34872d9d9"}
Nov 04 01:40:04.014745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:04.014685    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:40:04.131190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:04.131142    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-79c79ff55f-qc5qq" podStartSLOduration=2.472759263 podStartE2EDuration="8.131130655s" totalImagesPullingTime="5.658371392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:04.129935309 +0000 UTC m=+334.846122164" watchObservedRunningTime="2026-11-04 01:40:04.131130655 +0000 UTC m=+334.847317505"
Nov 04 01:40:04.247306 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:04.247158    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9" podStartSLOduration=2.547712825 podStartE2EDuration="8.247141354s" totalImagesPullingTime="5.699428529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:04.246615935 +0000 UTC m=+334.962802793" watchObservedRunningTime="2026-11-04 01:40:04.247141354 +0000 UTC m=+334.963328212"
Nov 04 01:40:07.105678 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:07.105645    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:07.349992 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:07.349964    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:15.021829 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:15.021792    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:40:15.022295 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:15.021874    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:40:15.190831 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:15.190421    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c79ff55f-qc5qq"]
Nov 04 01:40:15.190831 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:15.190662    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-79c79ff55f-qc5qq" podUID="42092908-969c-4e77-9419-941e9d084409" containerName="manager" containerID="cri-o://3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f" gracePeriod=30
Nov 04 01:40:15.969041 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:15.969017    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:40:16.021981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.021952    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h\" (UniqueName: \"kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h\") pod \"42092908-969c-4e77-9419-941e9d084409\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") "
Nov 04 01:40:16.022406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.022050    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\") pod \"42092908-969c-4e77-9419-941e9d084409\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") "
Nov 04 01:40:16.022406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.022115    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs\") pod \"42092908-969c-4e77-9419-941e9d084409\" (UID: \"42092908-969c-4e77-9419-941e9d084409\") "
Nov 04 01:40:16.024434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.024399    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs" pod "42092908-969c-4e77-9419-941e9d084409" (UID: "42092908-969c-4e77-9419-941e9d084409"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:40:16.024650 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.024629    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h" pod "42092908-969c-4e77-9419-941e9d084409" (UID: "42092908-969c-4e77-9419-941e9d084409"). InnerVolumeSpecName "kube-api-access-qls6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:40:16.024747 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.024646    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert" pod "42092908-969c-4e77-9419-941e9d084409" (UID: "42092908-969c-4e77-9419-941e9d084409"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:40:16.068874 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.068848    5039 generic.go:356] "Generic (PLEG): container finished" podID="42092908-969c-4e77-9419-941e9d084409" containerID="3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f" exitCode=0
Nov 04 01:40:16.069105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.068899    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c79ff55f-qc5qq"
Nov 04 01:40:16.069105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.068937    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c79ff55f-qc5qq" event={"ID":"42092908-969c-4e77-9419-941e9d084409","Type":"ContainerDied","Data":"3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f"}
Nov 04 01:40:16.069105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.068997    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c79ff55f-qc5qq" event={"ID":"42092908-969c-4e77-9419-941e9d084409","Type":"ContainerDied","Data":"6a08b9f4ff3842be937fb6c8626f676d307c6020cf505cee34bf674a0fcc42e8"}
Nov 04 01:40:16.069105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.069029    5039 scope.go:122] "RemoveContainer" containerID="3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f"
Nov 04 01:40:16.077987 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.077971    5039 scope.go:122] "RemoveContainer" containerID="3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f"
Nov 04 01:40:16.078221 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:40:16.078205    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f\": container with ID starting with 3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f not found: ID does not exist" containerID="3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f"
Nov 04 01:40:16.078268 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.078232    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f"} err="failed to get container status \"3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f\": rpc error: code = NotFound desc = could not find container \"3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f\": container with ID starting with 3825f4eea694eedb96a04f7c979684139290b89814b78cd5ae3da3271477df1f not found: ID does not exist"
Nov 04 01:40:16.122906 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.122877    5039 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-webhook-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:40:16.122906 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.122906    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42092908-969c-4e77-9419-941e9d084409-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:40:16.123037 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.122922    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qls6h\" (UniqueName: \"kubernetes.io/projected/42092908-969c-4e77-9419-941e9d084409-kube-api-access-qls6h\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:40:16.196529 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.196501    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c79ff55f-qc5qq"]
Nov 04 01:40:16.200412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:16.200391    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79c79ff55f-qc5qq"]
Nov 04 01:40:17.405252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:17.405219    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42092908-969c-4e77-9419-941e9d084409" path="/var/lib/kubelet/pods/42092908-969c-4e77-9419-941e9d084409/volumes"
Nov 04 01:40:21.462080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:21.462048    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:22.750758 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:40:22.750726    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:40:24.317189 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:24.317154    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:37.449383 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:37.449343    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:37.713187 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:37.713106    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:38.885031 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:38.884997    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:39.129025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:39.128994    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:40.395875 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:40.395848    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:40.643767 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:40.643726    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:40.891905 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:40.891873    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:41.142878 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.142790    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:41.166388 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.166359    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-gzdl5"]
Nov 04 01:40:41.166833 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.166814    5039 cpu_manager.go:414] "containerMap: removing container" podUID="42092908-969c-4e77-9419-941e9d084409" containerName="manager"
Nov 04 01:40:41.166833 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.166832    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42092908-969c-4e77-9419-941e9d084409" containerName="manager"
Nov 04 01:40:41.166978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.166911    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="42092908-969c-4e77-9419-941e9d084409" containerName="manager"
Nov 04 01:40:41.170037 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.170016    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.221639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.221619    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sxfzw\""
Nov 04 01:40:41.271175 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.271145    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 04 01:40:41.272851 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.272830    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-gzdl5"]
Nov 04 01:40:41.278224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.278204    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"]
Nov 04 01:40:41.281641 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.281627    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.333890 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.333870    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 04 01:40:41.348357 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.348335    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjltj\" (UniqueName: \"kubernetes.io/projected/8186cb9e-da9d-4f07-a3cd-a903e1cc7dce-kube-api-access-qjltj\") pod \"kserve-controller-manager-776cc66d47-gzdl5\" (UID: \"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.348459 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.348363    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8186cb9e-da9d-4f07-a3cd-a903e1cc7dce-cert\") pod \"kserve-controller-manager-776cc66d47-gzdl5\" (UID: \"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.377156 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.377138    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kgwm5\""
Nov 04 01:40:41.382798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.382780    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"]
Nov 04 01:40:41.395444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.395391    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:41.449445 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.449418    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pvn9\" (UniqueName: \"kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9\") pod \"llmisvc-controller-manager-869d5dcc5d-tkbx8\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.449857 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.449535    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjltj\" (UniqueName: \"kubernetes.io/projected/8186cb9e-da9d-4f07-a3cd-a903e1cc7dce-kube-api-access-qjltj\") pod \"kserve-controller-manager-776cc66d47-gzdl5\" (UID: \"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.449857 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.449599    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8186cb9e-da9d-4f07-a3cd-a903e1cc7dce-cert\") pod \"kserve-controller-manager-776cc66d47-gzdl5\" (UID: \"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.449857 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.449677    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tkbx8\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.452475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.452450    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8186cb9e-da9d-4f07-a3cd-a903e1cc7dce-cert\") pod \"kserve-controller-manager-776cc66d47-gzdl5\" (UID: \"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.510828 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.510806    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjltj\" (UniqueName: \"kubernetes.io/projected/8186cb9e-da9d-4f07-a3cd-a903e1cc7dce-kube-api-access-qjltj\") pod \"kserve-controller-manager-776cc66d47-gzdl5\" (UID: \"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce\") " pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.550636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.550608    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tkbx8\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.550727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.550651    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pvn9\" (UniqueName: \"kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9\") pod \"llmisvc-controller-manager-869d5dcc5d-tkbx8\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.552888 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.552869    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tkbx8\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.610983 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.610953    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pvn9\" (UniqueName: \"kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9\") pod \"llmisvc-controller-manager-869d5dcc5d-tkbx8\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.644358 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.644333    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:41.782959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.782904    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:41.893736 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.893710    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:41.990094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.990057    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"]
Nov 04 01:40:41.993677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:41.993646    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.055019 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.054911    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fhqrz\""
Nov 04 01:40:42.055166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.055035    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 01:40:42.055337 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.055302    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 01:40:42.056038 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.056018    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"]
Nov 04 01:40:42.156250 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.156221    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.156363 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.156255    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.156363 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.156342    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.156444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.156429    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbpsj\" (UniqueName: \"kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.175627 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.168962    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-gzdl5"]
Nov 04 01:40:42.187027 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.186997    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5" event={"ID":"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce","Type":"ContainerStarted","Data":"1112f9b165edd6f3971658664d75821f779d3a009c27605e235a2585b1864f03"}
Nov 04 01:40:42.257086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.257055    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.257212 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.257097    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.257212 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.257145    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.257325 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.257220    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbpsj\" (UniqueName: \"kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.257919 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.257832    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.259443 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.259422    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.259519 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.259475    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.262299 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.262280    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"]
Nov 04 01:40:42.264185 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:40:42.264167    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod63a9e470_1054_4eb8_b7b5_22c9ce01ab56.slice/crio-ca133d9282a605349fa321ace1d07c43c8fb26518e35c69e7f4bf085d0df677e WatchSource:0}: Error finding container ca133d9282a605349fa321ace1d07c43c8fb26518e35c69e7f4bf085d0df677e: Status 404 returned error can't find the container with id ca133d9282a605349fa321ace1d07c43c8fb26518e35c69e7f4bf085d0df677e
Nov 04 01:40:42.320462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.320416    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbpsj\" (UniqueName: \"kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj\") pod \"maas-api-55d5d89d45-cxlvt\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") " pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.606297 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.606275    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:42.636487 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.636458    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:42.980758 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:42.980736    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"]
Nov 04 01:40:43.192468 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:43.192434    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt" event={"ID":"c32e0146-d0ac-4105-af37-afb6d600b14d","Type":"ContainerStarted","Data":"52532fe34f7f9b58725ab6f7ab00c6058d0827a57c13605499c3af6fe59f73be"}
Nov 04 01:40:43.193964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:43.193938    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8" event={"ID":"63a9e470-1054-4eb8-b7b5-22c9ce01ab56","Type":"ContainerStarted","Data":"ca133d9282a605349fa321ace1d07c43c8fb26518e35c69e7f4bf085d0df677e"}
Nov 04 01:40:45.206997 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:45.206917    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5" event={"ID":"8186cb9e-da9d-4f07-a3cd-a903e1cc7dce","Type":"ContainerStarted","Data":"913b63332e6690b6a8733f25fa0f3e2efb0170bd99c92b6d630fbf3ccdfd60b0"}
Nov 04 01:40:45.207524 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:45.207065    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:40:47.797308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:47.797222    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:48.044594 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.042394    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:48.247866 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.247824    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8" event={"ID":"63a9e470-1054-4eb8-b7b5-22c9ce01ab56","Type":"ContainerStarted","Data":"dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8"}
Nov 04 01:40:48.248082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.247926    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:40:48.298821 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.298791    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:48.369819 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.369765    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8" podStartSLOduration=2.145059347 podStartE2EDuration="7.369750067s" totalImagesPullingTime="5.22469072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:48.368972908 +0000 UTC m=+379.085159763" watchObservedRunningTime="2026-11-04 01:40:48.369750067 +0000 UTC m=+379.085936902"
Nov 04 01:40:48.370724 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.370687    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5" podStartSLOduration=4.626151943 podStartE2EDuration="7.370677828s" totalImagesPullingTime="2.744525885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:45.346206181 +0000 UTC m=+376.062393037" watchObservedRunningTime="2026-11-04 01:40:48.370677828 +0000 UTC m=+379.086864684"
Nov 04 01:40:48.556773 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.556662    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:48.796885 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:48.796851    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:49.048803 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:49.048772    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:49.254177 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:49.254094    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt" event={"ID":"c32e0146-d0ac-4105-af37-afb6d600b14d","Type":"ContainerStarted","Data":"cb76b32a9a3921e209a53906b72c9de0c6e664c2ccd220168d32aa52b8ecc521"}
Nov 04 01:40:49.254329 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:49.254233    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:49.370889 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:49.370831    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt" podStartSLOduration=2.356755717 podStartE2EDuration="8.3708103s" totalImagesPullingTime="6.014054583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:49.370348898 +0000 UTC m=+380.086535752" watchObservedRunningTime="2026-11-04 01:40:49.3708103 +0000 UTC m=+380.086997156"
Nov 04 01:40:52.757156 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:40:52.757112    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:40:53.443891 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:53.443859    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:40:53.444060 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:53.443918    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 04 01:40:54.280639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:54.280606    5039 generic.go:356] "Generic (PLEG): container finished" podID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerID="d8ae395e374e8ed4ef47d998e84093b92e69163cef17c591caeb1f45ac1109b4" exitCode=0
Nov 04 01:40:54.281004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:54.280677    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" event={"ID":"dbeaf941-77d4-45f8-8e94-825323dc4a96","Type":"ContainerDied","Data":"d8ae395e374e8ed4ef47d998e84093b92e69163cef17c591caeb1f45ac1109b4"}
Nov 04 01:40:54.281045 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:54.281022    5039 scope.go:122] "RemoveContainer" containerID="d8ae395e374e8ed4ef47d998e84093b92e69163cef17c591caeb1f45ac1109b4"
Nov 04 01:40:54.282146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:54.282119    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:40:54.282214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:54.282186    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:40:54.282251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:54.282217    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:40:55.269562 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:55.269532    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:40:55.286718 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:55.286692    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" event={"ID":"dbeaf941-77d4-45f8-8e94-825323dc4a96","Type":"ContainerStarted","Data":"8c405a89171a7659c877828118c03716b7ec4dfc3aff755390107fdc29f89204"}
Nov 04 01:40:55.741598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:55.735937    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:40:55.741598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:55.736056    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:40:55.741598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:55.736096    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 04 01:40:55.948548 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:55.948512    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:56.156064 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.156030    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-lwm7l"]
Nov 04 01:40:56.159801 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.159781    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:40:56.194848 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.194816    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:56.209959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.209935    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbcc7\" (UniqueName: \"kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7\") pod \"authorino-6f6944bcdd-lwm7l\" (UID: \"84e715c4-bffd-4f18-a76f-d8cb02e60f44\") " pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:40:56.216911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.216881    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-lwm7l"]
Nov 04 01:40:56.310813 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.310786    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbcc7\" (UniqueName: \"kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7\") pod \"authorino-6f6944bcdd-lwm7l\" (UID: \"84e715c4-bffd-4f18-a76f-d8cb02e60f44\") " pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:40:56.371842 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.371815    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbcc7\" (UniqueName: \"kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7\") pod \"authorino-6f6944bcdd-lwm7l\" (UID: \"84e715c4-bffd-4f18-a76f-d8cb02e60f44\") " pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:40:56.471912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.471807    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:40:56.535594 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.532647    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-lwm7l"]
Nov 04 01:40:56.561423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.561385    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-659cdbc977-49rwz"]
Nov 04 01:40:56.567247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.566897    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:40:56.614177 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.614153    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbfjm\" (UniqueName: \"kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm\") pod \"authorino-659cdbc977-49rwz\" (UID: \"afdb3d8f-fba8-40ec-8912-e9c2be15509e\") " pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:40:56.617787 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.617761    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-659cdbc977-49rwz"]
Nov 04 01:40:56.714749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.714712    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbfjm\" (UniqueName: \"kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm\") pod \"authorino-659cdbc977-49rwz\" (UID: \"afdb3d8f-fba8-40ec-8912-e9c2be15509e\") " pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:40:56.747509 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.746989    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:40:56.747509 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.747051    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 04 01:40:56.775774 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.775742    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbfjm\" (UniqueName: \"kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm\") pod \"authorino-659cdbc977-49rwz\" (UID: \"afdb3d8f-fba8-40ec-8912-e9c2be15509e\") " pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:40:56.868377 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.868347    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-lwm7l"]
Nov 04 01:40:56.870474 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:40:56.870437    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84e715c4_bffd_4f18_a76f_d8cb02e60f44.slice/crio-c4b3a4d24e03cb5697a4c782490510ab56d339cb21b9f18cd337797bcd4e473a WatchSource:0}: Error finding container c4b3a4d24e03cb5697a4c782490510ab56d339cb21b9f18cd337797bcd4e473a: Status 404 returned error can't find the container with id c4b3a4d24e03cb5697a4c782490510ab56d339cb21b9f18cd337797bcd4e473a
Nov 04 01:40:56.880669 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.880649    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:40:56.993371 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:56.993334    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659cdbc977-49rwz"]
Nov 04 01:40:57.017945 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.017877    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9bdb54dbc-72v2t"]
Nov 04 01:40:57.045594 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.044153    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.079226 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.079198    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bdb54dbc-72v2t"]
Nov 04 01:40:57.096507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.096486    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 01:40:57.117966 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.117941    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64gsb\" (UniqueName: \"kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb\") pod \"authorino-9bdb54dbc-72v2t\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") " pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.118085 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.118067    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert\") pod \"authorino-9bdb54dbc-72v2t\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") " pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.219162 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.219137    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert\") pod \"authorino-9bdb54dbc-72v2t\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") " pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.219324 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.219177    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64gsb\" (UniqueName: \"kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb\") pod \"authorino-9bdb54dbc-72v2t\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") " pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.241591 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.240173    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert\") pod \"authorino-9bdb54dbc-72v2t\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") " pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.272493 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.272467    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659cdbc977-49rwz"]
Nov 04 01:40:57.274916 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:40:57.274890    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafdb3d8f_fba8_40ec_8912_e9c2be15509e.slice/crio-e6b70b7deda55b6f669cfc874115d6075fcb78990b376971379c1f001910bfd2 WatchSource:0}: Error finding container e6b70b7deda55b6f669cfc874115d6075fcb78990b376971379c1f001910bfd2: Status 404 returned error can't find the container with id e6b70b7deda55b6f669cfc874115d6075fcb78990b376971379c1f001910bfd2
Nov 04 01:40:57.279455 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.279435    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64gsb\" (UniqueName: \"kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb\") pod \"authorino-9bdb54dbc-72v2t\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") " pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.297266 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.297235    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l" event={"ID":"84e715c4-bffd-4f18-a76f-d8cb02e60f44","Type":"ContainerStarted","Data":"c4b3a4d24e03cb5697a4c782490510ab56d339cb21b9f18cd337797bcd4e473a"}
Nov 04 01:40:57.298393 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.298371    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659cdbc977-49rwz" event={"ID":"afdb3d8f-fba8-40ec-8912-e9c2be15509e","Type":"ContainerStarted","Data":"e6b70b7deda55b6f669cfc874115d6075fcb78990b376971379c1f001910bfd2"}
Nov 04 01:40:57.358125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.358097    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:40:57.733999 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.733971    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:40:57.734276 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.734246    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 04 01:40:57.749820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:57.748687    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bdb54dbc-72v2t"]
Nov 04 01:40:57.752995 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:40:57.752941    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaee683b_4071_48ce_a467_d8419b5f8274.slice/crio-85d92ae13ff4bcede24e9d28dc0a3f278f511e53245edf6991ce864934a412e4 WatchSource:0}: Error finding container 85d92ae13ff4bcede24e9d28dc0a3f278f511e53245edf6991ce864934a412e4: Status 404 returned error can't find the container with id 85d92ae13ff4bcede24e9d28dc0a3f278f511e53245edf6991ce864934a412e4
Nov 04 01:40:58.304433 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:58.304395    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bdb54dbc-72v2t" event={"ID":"aaee683b-4071-48ce-a467-d8419b5f8274","Type":"ContainerStarted","Data":"85d92ae13ff4bcede24e9d28dc0a3f278f511e53245edf6991ce864934a412e4"}
Nov 04 01:40:58.732590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:58.728919    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:40:58.732590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:58.728973    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 04 01:40:59.355603 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.355452    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l" event={"ID":"84e715c4-bffd-4f18-a76f-d8cb02e60f44","Type":"ContainerStarted","Data":"81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025"}
Nov 04 01:40:59.355948 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.355831    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l" podUID="84e715c4-bffd-4f18-a76f-d8cb02e60f44" containerName="authorino" containerID="cri-o://81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025" gracePeriod=30
Nov 04 01:40:59.359588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.359023    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659cdbc977-49rwz" event={"ID":"afdb3d8f-fba8-40ec-8912-e9c2be15509e","Type":"ContainerStarted","Data":"3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1"}
Nov 04 01:40:59.359588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.359144    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-659cdbc977-49rwz" podUID="afdb3d8f-fba8-40ec-8912-e9c2be15509e" containerName="authorino" containerID="cri-o://3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1" gracePeriod=30
Nov 04 01:40:59.367517 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.367488    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bdb54dbc-72v2t" event={"ID":"aaee683b-4071-48ce-a467-d8419b5f8274","Type":"ContainerStarted","Data":"d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82"}
Nov 04 01:40:59.471925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.471861    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l" podStartSLOduration=1.775653097 podStartE2EDuration="3.471842954s" totalImagesPullingTime="1.696189857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:59.46993915 +0000 UTC m=+390.186126007" watchObservedRunningTime="2026-11-04 01:40:59.471842954 +0000 UTC m=+390.188029810"
Nov 04 01:40:59.588072 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.588013    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-659cdbc977-49rwz" podStartSLOduration=2.161440175 podStartE2EDuration="3.58799304s" totalImagesPullingTime="1.426552865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:59.585774912 +0000 UTC m=+390.301961769" watchObservedRunningTime="2026-11-04 01:40:59.58799304 +0000 UTC m=+390.304179896"
Nov 04 01:40:59.708366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.708294    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-9bdb54dbc-72v2t" podStartSLOduration=2.48821055 podStartE2EDuration="3.708255111s" totalImagesPullingTime="1.220044561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:59.700664042 +0000 UTC m=+390.416850899" watchObservedRunningTime="2026-11-04 01:40:59.708255111 +0000 UTC m=+390.424441968"
Nov 04 01:40:59.728597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.727855    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bbc54"]
Nov 04 01:40:59.728597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.728068    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54" podUID="f4d3ee81-fb03-4e8a-adc5-9732d495e675" containerName="authorino" containerID="cri-o://22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06" gracePeriod=30
Nov 04 01:40:59.731725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.729481    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:40:59.731725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.729555    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Nov 04 01:40:59.767590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.762101    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:40:59.819165 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.819140    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:40:59.874422 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.874388    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7\" (UniqueName: \"kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7\") pod \"84e715c4-bffd-4f18-a76f-d8cb02e60f44\" (UID: \"84e715c4-bffd-4f18-a76f-d8cb02e60f44\") "
Nov 04 01:40:59.874620 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.874521    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm\" (UniqueName: \"kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm\") pod \"afdb3d8f-fba8-40ec-8912-e9c2be15509e\" (UID: \"afdb3d8f-fba8-40ec-8912-e9c2be15509e\") "
Nov 04 01:40:59.877247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.877212    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm" pod "afdb3d8f-fba8-40ec-8912-e9c2be15509e" (UID: "afdb3d8f-fba8-40ec-8912-e9c2be15509e"). InnerVolumeSpecName "kube-api-access-kbfjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:40:59.877643 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.877627    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7" pod "84e715c4-bffd-4f18-a76f-d8cb02e60f44" (UID: "84e715c4-bffd-4f18-a76f-d8cb02e60f44"). InnerVolumeSpecName "kube-api-access-tbcc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:40:59.975184 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.975146    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbfjm\" (UniqueName: \"kubernetes.io/projected/afdb3d8f-fba8-40ec-8912-e9c2be15509e-kube-api-access-kbfjm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:40:59.975184 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:40:59.975178    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbcc7\" (UniqueName: \"kubernetes.io/projected/84e715c4-bffd-4f18-a76f-d8cb02e60f44-kube-api-access-tbcc7\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:41:00.088668 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.088640    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:41:00.177989 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.177889    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg\" (UniqueName: \"kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg\") pod \"f4d3ee81-fb03-4e8a-adc5-9732d495e675\" (UID: \"f4d3ee81-fb03-4e8a-adc5-9732d495e675\") "
Nov 04 01:41:00.180304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.180271    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg" pod "f4d3ee81-fb03-4e8a-adc5-9732d495e675" (UID: "f4d3ee81-fb03-4e8a-adc5-9732d495e675"). InnerVolumeSpecName "kube-api-access-czhxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:41:00.278977 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.278943    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czhxg\" (UniqueName: \"kubernetes.io/projected/f4d3ee81-fb03-4e8a-adc5-9732d495e675-kube-api-access-czhxg\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:41:00.373814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.373737    5039 generic.go:356] "Generic (PLEG): container finished" podID="f4d3ee81-fb03-4e8a-adc5-9732d495e675" containerID="22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06" exitCode=0
Nov 04 01:41:00.374025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.373783    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54" event={"ID":"f4d3ee81-fb03-4e8a-adc5-9732d495e675","Type":"ContainerDied","Data":"22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06"}
Nov 04 01:41:00.374025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.373972    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54" event={"ID":"f4d3ee81-fb03-4e8a-adc5-9732d495e675","Type":"ContainerDied","Data":"cd53436f7f6f7aa6bb2f90030e4942434f93fca5447b058e0beb9e0b264138da"}
Nov 04 01:41:00.374025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.374000    5039 scope.go:122] "RemoveContainer" containerID="22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06"
Nov 04 01:41:00.374025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.373799    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bbc54"
Nov 04 01:41:00.375448 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.375428    5039 generic.go:356] "Generic (PLEG): container finished" podID="84e715c4-bffd-4f18-a76f-d8cb02e60f44" containerID="81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025" exitCode=0
Nov 04 01:41:00.375559 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.375468    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l"
Nov 04 01:41:00.375559 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.375519    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l" event={"ID":"84e715c4-bffd-4f18-a76f-d8cb02e60f44","Type":"ContainerDied","Data":"81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025"}
Nov 04 01:41:00.375699 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.375555    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-lwm7l" event={"ID":"84e715c4-bffd-4f18-a76f-d8cb02e60f44","Type":"ContainerDied","Data":"c4b3a4d24e03cb5697a4c782490510ab56d339cb21b9f18cd337797bcd4e473a"}
Nov 04 01:41:00.377601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.376950    5039 generic.go:356] "Generic (PLEG): container finished" podID="afdb3d8f-fba8-40ec-8912-e9c2be15509e" containerID="3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1" exitCode=0
Nov 04 01:41:00.377601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.376987    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659cdbc977-49rwz" event={"ID":"afdb3d8f-fba8-40ec-8912-e9c2be15509e","Type":"ContainerDied","Data":"3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1"}
Nov 04 01:41:00.377601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.377001    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659cdbc977-49rwz"
Nov 04 01:41:00.377601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.377022    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659cdbc977-49rwz" event={"ID":"afdb3d8f-fba8-40ec-8912-e9c2be15509e","Type":"ContainerDied","Data":"e6b70b7deda55b6f669cfc874115d6075fcb78990b376971379c1f001910bfd2"}
Nov 04 01:41:00.386020 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.385984    5039 scope.go:122] "RemoveContainer" containerID="22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06"
Nov 04 01:41:00.386367 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:41:00.386286    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06\": container with ID starting with 22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06 not found: ID does not exist" containerID="22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06"
Nov 04 01:41:00.386367 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.386317    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06"} err="failed to get container status \"22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06\": rpc error: code = NotFound desc = could not find container \"22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06\": container with ID starting with 22c1f8e83d6fbd0b26ccb90ecf6d2a6f68ff1b2d98d9720d78e486184f113d06 not found: ID does not exist"
Nov 04 01:41:00.386367 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.386341    5039 scope.go:122] "RemoveContainer" containerID="81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025"
Nov 04 01:41:00.394781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.394763    5039 scope.go:122] "RemoveContainer" containerID="81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025"
Nov 04 01:41:00.395032 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:41:00.395015    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025\": container with ID starting with 81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025 not found: ID does not exist" containerID="81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025"
Nov 04 01:41:00.395086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.395039    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025"} err="failed to get container status \"81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025\": rpc error: code = NotFound desc = could not find container \"81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025\": container with ID starting with 81ceea0c51c89f7a010488f6ee086b033907369b7a9c3f607f31df4a56e7a025 not found: ID does not exist"
Nov 04 01:41:00.395086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.395056    5039 scope.go:122] "RemoveContainer" containerID="3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1"
Nov 04 01:41:00.402490 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.402466    5039 scope.go:122] "RemoveContainer" containerID="3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1"
Nov 04 01:41:00.404597 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:41:00.403096    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1\": container with ID starting with 3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1 not found: ID does not exist" containerID="3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1"
Nov 04 01:41:00.404597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.403131    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1"} err="failed to get container status \"3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1\": rpc error: code = NotFound desc = could not find container \"3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1\": container with ID starting with 3982179812cf5611d90ebca483a563e6639e0a30616b065df39efd74f916c1a1 not found: ID does not exist"
Nov 04 01:41:00.535713 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.535635    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bbc54"]
Nov 04 01:41:00.544918 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.542898    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bbc54"]
Nov 04 01:41:00.649543 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.649519    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-lwm7l"]
Nov 04 01:41:00.653338 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.653316    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-lwm7l"]
Nov 04 01:41:00.732952 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.732925    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:41:00.733233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.733208    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:41:00.733947 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.733930    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88"
Nov 04 01:41:00.760860 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.760835    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659cdbc977-49rwz"]
Nov 04 01:41:00.764660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:00.764641    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-659cdbc977-49rwz"]
Nov 04 01:41:01.411087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:01.411057    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84e715c4-bffd-4f18-a76f-d8cb02e60f44" path="/var/lib/kubelet/pods/84e715c4-bffd-4f18-a76f-d8cb02e60f44/volumes"
Nov 04 01:41:01.411462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:01.411358    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afdb3d8f-fba8-40ec-8912-e9c2be15509e" path="/var/lib/kubelet/pods/afdb3d8f-fba8-40ec-8912-e9c2be15509e/volumes"
Nov 04 01:41:01.411659 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:01.411647    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4d3ee81-fb03-4e8a-adc5-9732d495e675" path="/var/lib/kubelet/pods/f4d3ee81-fb03-4e8a-adc5-9732d495e675/volumes"
Nov 04 01:41:06.280839 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.280804    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"]
Nov 04 01:41:06.281282 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.281008    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt" podUID="c32e0146-d0ac-4105-af37-afb6d600b14d" containerName="maas-api" containerID="cri-o://cb76b32a9a3921e209a53906b72c9de0c6e664c2ccd220168d32aa52b8ecc521" gracePeriod=30
Nov 04 01:41:06.412040 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.412012    5039 generic.go:356] "Generic (PLEG): container finished" podID="c32e0146-d0ac-4105-af37-afb6d600b14d" containerID="cb76b32a9a3921e209a53906b72c9de0c6e664c2ccd220168d32aa52b8ecc521" exitCode=0
Nov 04 01:41:06.412175 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.412059    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt" event={"ID":"c32e0146-d0ac-4105-af37-afb6d600b14d","Type":"ContainerDied","Data":"cb76b32a9a3921e209a53906b72c9de0c6e664c2ccd220168d32aa52b8ecc521"}
Nov 04 01:41:06.528217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.528194    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:41:06.636520 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.636495    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca\") pod \"c32e0146-d0ac-4105-af37-afb6d600b14d\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") "
Nov 04 01:41:06.636685 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.636618    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj\" (UniqueName: \"kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj\") pod \"c32e0146-d0ac-4105-af37-afb6d600b14d\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") "
Nov 04 01:41:06.636746 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.636719    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs\") pod \"c32e0146-d0ac-4105-af37-afb6d600b14d\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") "
Nov 04 01:41:06.636805 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.636753    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls\") pod \"c32e0146-d0ac-4105-af37-afb6d600b14d\" (UID: \"c32e0146-d0ac-4105-af37-afb6d600b14d\") "
Nov 04 01:41:06.636906 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.636885    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca" pod "c32e0146-d0ac-4105-af37-afb6d600b14d" (UID: "c32e0146-d0ac-4105-af37-afb6d600b14d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:41:06.637003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.636985    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c32e0146-d0ac-4105-af37-afb6d600b14d-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:41:06.638956 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.638925    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls" pod "c32e0146-d0ac-4105-af37-afb6d600b14d" (UID: "c32e0146-d0ac-4105-af37-afb6d600b14d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:41:06.638956 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.638939    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs" pod "c32e0146-d0ac-4105-af37-afb6d600b14d" (UID: "c32e0146-d0ac-4105-af37-afb6d600b14d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:41:06.639070 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.638935    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj" pod "c32e0146-d0ac-4105-af37-afb6d600b14d" (UID: "c32e0146-d0ac-4105-af37-afb6d600b14d"). InnerVolumeSpecName "kube-api-access-qbpsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:41:06.737737 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.737710    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:41:06.737737 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.737735    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c32e0146-d0ac-4105-af37-afb6d600b14d-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:41:06.737737 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:06.737745    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbpsj\" (UniqueName: \"kubernetes.io/projected/c32e0146-d0ac-4105-af37-afb6d600b14d-kube-api-access-qbpsj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:41:07.417334 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:07.417310    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"
Nov 04 01:41:07.417795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:07.417332    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt" event={"ID":"c32e0146-d0ac-4105-af37-afb6d600b14d","Type":"ContainerDied","Data":"52532fe34f7f9b58725ab6f7ab00c6058d0827a57c13605499c3af6fe59f73be"}
Nov 04 01:41:07.417795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:07.417393    5039 scope.go:122] "RemoveContainer" containerID="cb76b32a9a3921e209a53906b72c9de0c6e664c2ccd220168d32aa52b8ecc521"
Nov 04 01:41:07.580677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:07.580650    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"]
Nov 04 01:41:07.629974 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:07.629942    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-55d5d89d45-cxlvt"]
Nov 04 01:41:09.405507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:09.405476    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c32e0146-d0ac-4105-af37-afb6d600b14d" path="/var/lib/kubelet/pods/c32e0146-d0ac-4105-af37-afb6d600b14d/volumes"
Nov 04 01:41:16.216683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:16.216649    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-gzdl5"
Nov 04 01:41:19.256721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:19.256692    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:41:21.809059 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:41:21.809008    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:41:22.751327 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:41:22.751290    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:41:52.751378 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:41:52.751342    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:42:01.257052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257021    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gstds"]
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257394    5039 cpu_manager.go:414] "containerMap: removing container" podUID="84e715c4-bffd-4f18-a76f-d8cb02e60f44" containerName="authorino"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257405    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84e715c4-bffd-4f18-a76f-d8cb02e60f44" containerName="authorino"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257433    5039 cpu_manager.go:414] "containerMap: removing container" podUID="afdb3d8f-fba8-40ec-8912-e9c2be15509e" containerName="authorino"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257437    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afdb3d8f-fba8-40ec-8912-e9c2be15509e" containerName="authorino"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257446    5039 cpu_manager.go:414] "containerMap: removing container" podUID="f4d3ee81-fb03-4e8a-adc5-9732d495e675" containerName="authorino"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257451    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4d3ee81-fb03-4e8a-adc5-9732d495e675" containerName="authorino"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257465    5039 cpu_manager.go:414] "containerMap: removing container" podUID="c32e0146-d0ac-4105-af37-afb6d600b14d" containerName="maas-api"
Nov 04 01:42:01.257506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257470    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c32e0146-d0ac-4105-af37-afb6d600b14d" containerName="maas-api"
Nov 04 01:42:01.257814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257526    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="c32e0146-d0ac-4105-af37-afb6d600b14d" containerName="maas-api"
Nov 04 01:42:01.257814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257534    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="84e715c4-bffd-4f18-a76f-d8cb02e60f44" containerName="authorino"
Nov 04 01:42:01.257814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257543    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="afdb3d8f-fba8-40ec-8912-e9c2be15509e" containerName="authorino"
Nov 04 01:42:01.257814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.257549    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4d3ee81-fb03-4e8a-adc5-9732d495e675" containerName="authorino"
Nov 04 01:42:01.260761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.260746    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.319154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.319122    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gstds"]
Nov 04 01:42:01.369295 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.369271    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d3c9ac2-e931-4dd6-8871-e4bfdf719c05-cert\") pod \"llmisvc-controller-manager-6466c554d8-gstds\" (UID: \"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.369413 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.369332    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h9zh\" (UniqueName: \"kubernetes.io/projected/5d3c9ac2-e931-4dd6-8871-e4bfdf719c05-kube-api-access-8h9zh\") pod \"llmisvc-controller-manager-6466c554d8-gstds\" (UID: \"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.469852 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.469826    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h9zh\" (UniqueName: \"kubernetes.io/projected/5d3c9ac2-e931-4dd6-8871-e4bfdf719c05-kube-api-access-8h9zh\") pod \"llmisvc-controller-manager-6466c554d8-gstds\" (UID: \"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.470000 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.469864    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d3c9ac2-e931-4dd6-8871-e4bfdf719c05-cert\") pod \"llmisvc-controller-manager-6466c554d8-gstds\" (UID: \"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.472339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.472316    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d3c9ac2-e931-4dd6-8871-e4bfdf719c05-cert\") pod \"llmisvc-controller-manager-6466c554d8-gstds\" (UID: \"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.530598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.530516    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h9zh\" (UniqueName: \"kubernetes.io/projected/5d3c9ac2-e931-4dd6-8871-e4bfdf719c05-kube-api-access-8h9zh\") pod \"llmisvc-controller-manager-6466c554d8-gstds\" (UID: \"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.573146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.573126    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:01.941651 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:01.941629    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gstds"]
Nov 04 01:42:01.943559 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:42:01.943526    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3c9ac2_e931_4dd6_8871_e4bfdf719c05.slice/crio-603d7246da5ce4d1a86c0857801def252ababac84655139d12bb13f4ef858b2e WatchSource:0}: Error finding container 603d7246da5ce4d1a86c0857801def252ababac84655139d12bb13f4ef858b2e: Status 404 returned error can't find the container with id 603d7246da5ce4d1a86c0857801def252ababac84655139d12bb13f4ef858b2e
Nov 04 01:42:02.686403 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:02.686374    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds" event={"ID":"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05","Type":"ContainerStarted","Data":"603d7246da5ce4d1a86c0857801def252ababac84655139d12bb13f4ef858b2e"}
Nov 04 01:42:03.692214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:03.692178    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds" event={"ID":"5d3c9ac2-e931-4dd6-8871-e4bfdf719c05","Type":"ContainerStarted","Data":"71332cb7c557da8e387734384bdc6617d2c8acd4d5545fb2c67fe6d28b9cba8a"}
Nov 04 01:42:03.692620 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:03.692253    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:03.807998 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:03.807946    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds" podStartSLOduration=2.069762198 podStartE2EDuration="2.8079321s" totalImagesPullingTime="738.169902ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:42:03.806291015 +0000 UTC m=+454.522477875" watchObservedRunningTime="2026-11-04 01:42:03.8079321 +0000 UTC m=+454.524118955"
Nov 04 01:42:22.152961 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:22.152928    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:22.751051 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:42:22.751018    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:42:34.699906 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:34.699877    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gstds"
Nov 04 01:42:34.865960 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:34.865930    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"]
Nov 04 01:42:34.866136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:34.866115    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8" podUID="63a9e470-1054-4eb8-b7b5-22c9ce01ab56" containerName="manager" containerID="cri-o://dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8" gracePeriod=30
Nov 04 01:42:35.118260 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.118236    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:42:35.248673 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.248589    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert\" (UniqueName: \"kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert\") pod \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") "
Nov 04 01:42:35.248673 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.248635    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9\" (UniqueName: \"kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9\") pod \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\" (UID: \"63a9e470-1054-4eb8-b7b5-22c9ce01ab56\") "
Nov 04 01:42:35.250768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.250739    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert" pod "63a9e470-1054-4eb8-b7b5-22c9ce01ab56" (UID: "63a9e470-1054-4eb8-b7b5-22c9ce01ab56"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:42:35.250892 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.250817    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9" pod "63a9e470-1054-4eb8-b7b5-22c9ce01ab56" (UID: "63a9e470-1054-4eb8-b7b5-22c9ce01ab56"). InnerVolumeSpecName "kube-api-access-6pvn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:42:35.349426 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.349398    5039 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:42:35.349588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.349431    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pvn9\" (UniqueName: \"kubernetes.io/projected/63a9e470-1054-4eb8-b7b5-22c9ce01ab56-kube-api-access-6pvn9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:42:35.846520 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.846488    5039 generic.go:356] "Generic (PLEG): container finished" podID="63a9e470-1054-4eb8-b7b5-22c9ce01ab56" containerID="dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8" exitCode=0
Nov 04 01:42:35.846941 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.846533    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"
Nov 04 01:42:35.846941 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.846584    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8" event={"ID":"63a9e470-1054-4eb8-b7b5-22c9ce01ab56","Type":"ContainerDied","Data":"dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8"}
Nov 04 01:42:35.846941 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.846636    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8" event={"ID":"63a9e470-1054-4eb8-b7b5-22c9ce01ab56","Type":"ContainerDied","Data":"ca133d9282a605349fa321ace1d07c43c8fb26518e35c69e7f4bf085d0df677e"}
Nov 04 01:42:35.846941 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.846658    5039 scope.go:122] "RemoveContainer" containerID="dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8"
Nov 04 01:42:35.855117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.855101    5039 scope.go:122] "RemoveContainer" containerID="dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8"
Nov 04 01:42:35.855374 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:42:35.855354    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8\": container with ID starting with dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8 not found: ID does not exist" containerID="dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8"
Nov 04 01:42:35.855442 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:35.855384    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8"} err="failed to get container status \"dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8\": rpc error: code = NotFound desc = could not find container \"dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8\": container with ID starting with dae156a6e13d8bf91bfc9d2fc569d58cbf97698f16407a8db830dda5edacd5f8 not found: ID does not exist"
Nov 04 01:42:36.008959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.008928    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"]
Nov 04 01:42:36.058089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.058062    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8"]
Nov 04 01:42:36.243981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.243890    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:36.493923 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.493894    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:36.853768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.853740    5039 generic.go:356] "Generic (PLEG): container finished" podID="2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4" containerID="32624751eedd57b3073dfc05cd5a289684b891838f5e167bbf23feb34872d9d9" exitCode=1
Nov 04 01:42:36.854150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.853788    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9" event={"ID":"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4","Type":"ContainerDied","Data":"32624751eedd57b3073dfc05cd5a289684b891838f5e167bbf23feb34872d9d9"}
Nov 04 01:42:36.854150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:36.854089    5039 scope.go:122] "RemoveContainer" containerID="32624751eedd57b3073dfc05cd5a289684b891838f5e167bbf23feb34872d9d9"
Nov 04 01:42:37.043668 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:37.043638    5039 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:42:37.405421 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:37.405391    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63a9e470-1054-4eb8-b7b5-22c9ce01ab56" path="/var/lib/kubelet/pods/63a9e470-1054-4eb8-b7b5-22c9ce01ab56/volumes"
Nov 04 01:42:38.867158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:38.867126    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9" event={"ID":"2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4","Type":"ContainerStarted","Data":"605d665e2f9f8343593376faee983fb818ed9fac6a494383b88e3ce8d6ac5c52"}
Nov 04 01:42:38.867662 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:38.867212    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:42:44.165262 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:44.165225    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:44.411437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:44.411404    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:49.874423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:42:49.874393    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8677bb4cb4-5d7s9"
Nov 04 01:42:52.750708 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:42:52.750673    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:43:08.345701 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:08.345658    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:08.906209 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:08.906178    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:10.337073 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:10.337043    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:10.601276 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:10.601244    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:17.385361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:17.385330    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:22.496292 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:22.496260    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:22.750292 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:43:22.750247    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:43:25.570770 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:25.570729    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:29.020094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:29.020058    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:32.943023 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:32.942971    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:36.050032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:36.049984    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:40.698201 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:40.698169    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:44.305537 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:44.305497    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:49.742426 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:43:49.742395    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:52.750882 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:43:52.750844    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:44:15.863071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.862994    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65457c7d56-kdp2h"]
Nov 04 01:44:15.863545 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.863435    5039 cpu_manager.go:414] "containerMap: removing container" podUID="63a9e470-1054-4eb8-b7b5-22c9ce01ab56" containerName="manager"
Nov 04 01:44:15.863545 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.863447    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63a9e470-1054-4eb8-b7b5-22c9ce01ab56" containerName="manager"
Nov 04 01:44:15.863545 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.863516    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="63a9e470-1054-4eb8-b7b5-22c9ce01ab56" containerName="manager"
Nov 04 01:44:15.866516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.866495    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:15.924297 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.924264    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65457c7d56-kdp2h"]
Nov 04 01:44:15.934304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.934274    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c5np\" (UniqueName: \"kubernetes.io/projected/ae5536f6-a4af-4705-b5ea-8177e0bfead1-kube-api-access-9c5np\") pod \"authorino-65457c7d56-kdp2h\" (UID: \"ae5536f6-a4af-4705-b5ea-8177e0bfead1\") " pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:15.934423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:15.934376    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae5536f6-a4af-4705-b5ea-8177e0bfead1-tls-cert\") pod \"authorino-65457c7d56-kdp2h\" (UID: \"ae5536f6-a4af-4705-b5ea-8177e0bfead1\") " pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:16.035378 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.035331    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c5np\" (UniqueName: \"kubernetes.io/projected/ae5536f6-a4af-4705-b5ea-8177e0bfead1-kube-api-access-9c5np\") pod \"authorino-65457c7d56-kdp2h\" (UID: \"ae5536f6-a4af-4705-b5ea-8177e0bfead1\") " pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:16.035550 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.035427    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae5536f6-a4af-4705-b5ea-8177e0bfead1-tls-cert\") pod \"authorino-65457c7d56-kdp2h\" (UID: \"ae5536f6-a4af-4705-b5ea-8177e0bfead1\") " pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:16.037961 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.037933    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae5536f6-a4af-4705-b5ea-8177e0bfead1-tls-cert\") pod \"authorino-65457c7d56-kdp2h\" (UID: \"ae5536f6-a4af-4705-b5ea-8177e0bfead1\") " pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:16.095973 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.095941    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c5np\" (UniqueName: \"kubernetes.io/projected/ae5536f6-a4af-4705-b5ea-8177e0bfead1-kube-api-access-9c5np\") pod \"authorino-65457c7d56-kdp2h\" (UID: \"ae5536f6-a4af-4705-b5ea-8177e0bfead1\") " pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:16.177192 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.177129    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65457c7d56-kdp2h"
Nov 04 01:44:16.548012 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.547960    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65457c7d56-kdp2h"]
Nov 04 01:44:16.550305 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:44:16.550277    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae5536f6_a4af_4705_b5ea_8177e0bfead1.slice/crio-198bfee9c256cf7ac979a476e4fea3dc7fe68cf121a4291ee2d620f00dcfdb5c WatchSource:0}: Error finding container 198bfee9c256cf7ac979a476e4fea3dc7fe68cf121a4291ee2d620f00dcfdb5c: Status 404 returned error can't find the container with id 198bfee9c256cf7ac979a476e4fea3dc7fe68cf121a4291ee2d620f00dcfdb5c
Nov 04 01:44:16.668962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.668937    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:16.934438 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:16.934410    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:17.365943 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:17.365914    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65457c7d56-kdp2h" event={"ID":"ae5536f6-a4af-4705-b5ea-8177e0bfead1","Type":"ContainerStarted","Data":"198bfee9c256cf7ac979a476e4fea3dc7fe68cf121a4291ee2d620f00dcfdb5c"}
Nov 04 01:44:18.374322 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.374287    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65457c7d56-kdp2h" event={"ID":"ae5536f6-a4af-4705-b5ea-8177e0bfead1","Type":"ContainerStarted","Data":"3d1cca94420e9f329d818f231934aff69554abffe7ab38c56b081b1f80ae5a4c"}
Nov 04 01:44:18.429802 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.429764    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:18.488975 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.488925    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-65457c7d56-kdp2h" podStartSLOduration=2.09637709 podStartE2EDuration="3.488912695s" totalImagesPullingTime="1.392535605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:44:18.487981086 +0000 UTC m=+589.204167943" watchObservedRunningTime="2026-11-04 01:44:18.488912695 +0000 UTC m=+589.205099550"
Nov 04 01:44:18.517002 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.516976    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9bdb54dbc-72v2t"]
Nov 04 01:44:18.517206 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.517185    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-9bdb54dbc-72v2t" podUID="aaee683b-4071-48ce-a467-d8419b5f8274" containerName="authorino" containerID="cri-o://d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82" gracePeriod=30
Nov 04 01:44:18.690139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.690104    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:18.768632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.768602    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:44:18.855649 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.855611    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb\" (UniqueName: \"kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb\") pod \"aaee683b-4071-48ce-a467-d8419b5f8274\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") "
Nov 04 01:44:18.855819 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.855686    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert\" (UniqueName: \"kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert\") pod \"aaee683b-4071-48ce-a467-d8419b5f8274\" (UID: \"aaee683b-4071-48ce-a467-d8419b5f8274\") "
Nov 04 01:44:18.857845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.857813    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb" pod "aaee683b-4071-48ce-a467-d8419b5f8274" (UID: "aaee683b-4071-48ce-a467-d8419b5f8274"). InnerVolumeSpecName "kube-api-access-64gsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:44:18.867224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.867199    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert" pod "aaee683b-4071-48ce-a467-d8419b5f8274" (UID: "aaee683b-4071-48ce-a467-d8419b5f8274"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:44:18.956959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.956925    5039 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaee683b-4071-48ce-a467-d8419b5f8274-tls-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:44:18.956959 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:18.956959    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64gsb\" (UniqueName: \"kubernetes.io/projected/aaee683b-4071-48ce-a467-d8419b5f8274-kube-api-access-64gsb\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:44:19.381004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.380962    5039 generic.go:356] "Generic (PLEG): container finished" podID="aaee683b-4071-48ce-a467-d8419b5f8274" containerID="d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82" exitCode=0
Nov 04 01:44:19.381483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.381007    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bdb54dbc-72v2t"
Nov 04 01:44:19.381483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.381037    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bdb54dbc-72v2t" event={"ID":"aaee683b-4071-48ce-a467-d8419b5f8274","Type":"ContainerDied","Data":"d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82"}
Nov 04 01:44:19.381483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.381094    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bdb54dbc-72v2t" event={"ID":"aaee683b-4071-48ce-a467-d8419b5f8274","Type":"ContainerDied","Data":"85d92ae13ff4bcede24e9d28dc0a3f278f511e53245edf6991ce864934a412e4"}
Nov 04 01:44:19.381483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.381127    5039 scope.go:122] "RemoveContainer" containerID="d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82"
Nov 04 01:44:19.390409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.390388    5039 scope.go:122] "RemoveContainer" containerID="d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82"
Nov 04 01:44:19.390667 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:44:19.390649    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82\": container with ID starting with d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82 not found: ID does not exist" containerID="d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82"
Nov 04 01:44:19.390730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.390676    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82"} err="failed to get container status \"d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82\": rpc error: code = NotFound desc = could not find container \"d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82\": container with ID starting with d75fa76ad26550171eddd6cbd5c1ad0191c3c98077fa3c052fa56e3f2351ba82 not found: ID does not exist"
Nov 04 01:44:19.496044 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.496005    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"]
Nov 04 01:44:19.496675 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.496650    5039 cpu_manager.go:414] "containerMap: removing container" podUID="aaee683b-4071-48ce-a467-d8419b5f8274" containerName="authorino"
Nov 04 01:44:19.496675 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.496675    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaee683b-4071-48ce-a467-d8419b5f8274" containerName="authorino"
Nov 04 01:44:19.496822 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.496794    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaee683b-4071-48ce-a467-d8419b5f8274" containerName="authorino"
Nov 04 01:44:19.500216 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.500197    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.543217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.543184    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9bdb54dbc-72v2t"]
Nov 04 01:44:19.548373 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.548346    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9bdb54dbc-72v2t"]
Nov 04 01:44:19.562294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562266    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562427 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562310    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562427 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562345    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562473    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562543 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562507    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562543 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562532    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562658 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562587    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvzqj\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562699    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.562777 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.562740    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.599971 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.599937    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"]
Nov 04 01:44:19.664046 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.663955    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664046 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664030    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664055    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664076    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664102    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvzqj\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664138    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664164    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664237    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664616 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664283    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664616 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664512    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664740 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664652    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664813    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.664945 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.664911    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.665082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.665061    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.667101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.667074    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.667520 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.667499    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.725523 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.725432    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvzqj\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.726914 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.726889    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8wpzd\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:19.814127 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:19.814093    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:20.241013 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.240977    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"]
Nov 04 01:44:20.242610 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:44:20.242553    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode100a836_e762_49ab_89c2_b82c0af447e9.slice/crio-b47e36b7334a9880fb3596ebe8a685e6c32a5e7d80ef401d3daa95fc486a17e6 WatchSource:0}: Error finding container b47e36b7334a9880fb3596ebe8a685e6c32a5e7d80ef401d3daa95fc486a17e6: Status 404 returned error can't find the container with id b47e36b7334a9880fb3596ebe8a685e6c32a5e7d80ef401d3daa95fc486a17e6
Nov 04 01:44:20.245227 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.245194    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:20.245310 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.245260    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:20.245310 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.245289    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:20.390934 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.390885    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" event={"ID":"e100a836-e762-49ab-89c2-b82c0af447e9","Type":"ContainerStarted","Data":"8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32"}
Nov 04 01:44:20.391428 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.390952    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" event={"ID":"e100a836-e762-49ab-89c2-b82c0af447e9","Type":"ContainerStarted","Data":"b47e36b7334a9880fb3596ebe8a685e6c32a5e7d80ef401d3daa95fc486a17e6"}
Nov 04 01:44:20.514267 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.514146    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podStartSLOduration=1.514130199 podStartE2EDuration="1.514130199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:44:20.512886114 +0000 UTC m=+591.229072969" watchObservedRunningTime="2026-11-04 01:44:20.514130199 +0000 UTC m=+591.230317055"
Nov 04 01:44:20.814268 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:20.814190    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:21.406549 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:21.406515    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaee683b-4071-48ce-a467-d8419b5f8274" path="/var/lib/kubelet/pods/aaee683b-4071-48ce-a467-d8419b5f8274/volumes"
Nov 04 01:44:21.815726 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:21.815654    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8wpzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 01:44:21.815726 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:21.815709    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 01:44:22.768857 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:44:22.768822    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:44:22.814667 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:22.814632    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8wpzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 01:44:22.814803 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:22.814689    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 01:44:22.877086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:22.877050    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:23.815490 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:23.815457    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8wpzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 01:44:23.815898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:23.815514    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 01:44:24.814436 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:24.814410    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8wpzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 01:44:24.814636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:24.814464    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 01:44:25.815285 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:25.815252    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8wpzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 01:44:25.815676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:25.815303    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 01:44:26.817964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:26.817920    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8wpzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Nov 04 01:44:26.818463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:26.817994    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Nov 04 01:44:28.688065 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:28.688022    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:28.688622 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:28.688588    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:28.689299 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:28.689276    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:44:29.450429 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:29.450405    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:44:29.452654 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:29.452623    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:44:37.922988 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:37.922954    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:38.172453 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:38.172416    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:52.750291 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:44:52.750257    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:44:59.054725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:59.054687    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:59.315346 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:44:59.315268    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:06.757434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:06.757400    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:07.027441 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:07.027365    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:07.453354 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:07.453320    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:07.709356 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:07.709271    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:07.949311 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:07.949280    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:08.201162 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:08.201131    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:08.514853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:08.514775    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:09.034726 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:09.034692    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:10.254798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:10.254766    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:10.517120 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:10.517043    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:11.972627 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:11.972592    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:12.231853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:12.231778    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:12.667426 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:12.667389    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:12.922237 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:12.922145    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:13.300947 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:13.300868    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:13.552022 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:13.551939    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:14.181511 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:14.181477    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:14.424104 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:14.424068    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:14.691047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:14.691016    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:15.294676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:15.294635    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:15.539950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:15.539907    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:15.791988 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:15.791925    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:16.049395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.049282    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:16.170045 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.170017    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"]
Nov 04 01:45:16.173657 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.173642    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.274921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.274897    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-f4d202-dockercfg-vdngc\""
Nov 04 01:45:16.274921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.274907    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-f4d202\""
Nov 04 01:45:16.276809 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.276791    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"]
Nov 04 01:45:16.278316 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.278299    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.278362 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.278329    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9k9\" (UniqueName: \"kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9\") pod \"payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.379736 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.379711    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9k9\" (UniqueName: \"kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9\") pod \"payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.379860 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.379810    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.380509 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.380491    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.440022 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.440000    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9k9\" (UniqueName: \"kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9\") pod \"payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.486244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.486225    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:16.858283 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.858161    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"]
Nov 04 01:45:16.860709 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:45:16.860681    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b4910a_7de8_4924_b37c_e98b8b324615.slice/crio-1c8d52c9d025afc70a948a9423d8bc3c08a1594c3958c3861a6d33c16e1dd357 WatchSource:0}: Error finding container 1c8d52c9d025afc70a948a9423d8bc3c08a1594c3958c3861a6d33c16e1dd357: Status 404 returned error can't find the container with id 1c8d52c9d025afc70a948a9423d8bc3c08a1594c3958c3861a6d33c16e1dd357
Nov 04 01:45:16.862701 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:16.862687    5039 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:45:17.672917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:17.672880    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" event={"ID":"65b4910a-7de8-4924-b37c-e98b8b324615","Type":"ContainerStarted","Data":"1c8d52c9d025afc70a948a9423d8bc3c08a1594c3958c3861a6d33c16e1dd357"}
Nov 04 01:45:17.716365 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:17.716338    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:17.960065 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:17.959997    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:18.276880 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.276767    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:18.501826 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.501795    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"]
Nov 04 01:45:18.505645 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.505624    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.557549 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.557488    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-82706f-serving-cert\""
Nov 04 01:45:18.596027 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.595999    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.596182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.596066    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.596182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.596108    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f4g9\" (UniqueName: \"kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.596269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.596238    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.606930 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.606890    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fhqrz\""
Nov 04 01:45:18.607052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.606998    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 01:45:18.609346 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.609310    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"]
Nov 04 01:45:18.697124 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.697096    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.697124 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.697128    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f4g9\" (UniqueName: \"kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.697376 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.697177    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.697376 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.697242    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.697834 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.697813    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.697919 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:18.697899    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-serving-cert: secret "maas-api-e2e-worker-w2-82706f-serving-cert" not found
Nov 04 01:45:18.697976 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:18.697959    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls podName:d2ae9460-8050-48a0-9632-7187c4a98fc5 nodeName:}" failed. No retries permitted until 2026-11-04 01:45:19.197946336 +0000 UTC m=+649.914133170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls") pod "maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" (UID: "d2ae9460-8050-48a0-9632-7187c4a98fc5") : secret "maas-api-e2e-worker-w2-82706f-serving-cert" not found
Nov 04 01:45:18.701046 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.701021    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.758799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.758772    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f4g9\" (UniqueName: \"kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:18.789306 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:18.789259    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:19.039214 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.039183    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:19.203346 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.203305    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:19.206122 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.206093    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:19.284768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.284730    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:19.421771 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.421739    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:19.685345 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.685264    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" event={"ID":"65b4910a-7de8-4924-b37c-e98b8b324615","Type":"ContainerStarted","Data":"df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d"}
Nov 04 01:45:19.685508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.685395    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:19.800523 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.800469    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" podStartSLOduration=1.219698141 podStartE2EDuration="3.800455666s" totalImagesPullingTime="2.580757525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:19.79963705 +0000 UTC m=+650.515823905" watchObservedRunningTime="2026-11-04 01:45:19.800455666 +0000 UTC m=+650.516642555"
Nov 04 01:45:19.848393 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:45:19.848358    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ae9460_8050_48a0_9632_7187c4a98fc5.slice/crio-3cfcd092902cf9f6e51c9d6729b16b1deb62cbe7d3689c616aaf402355356fde WatchSource:0}: Error finding container 3cfcd092902cf9f6e51c9d6729b16b1deb62cbe7d3689c616aaf402355356fde: Status 404 returned error can't find the container with id 3cfcd092902cf9f6e51c9d6729b16b1deb62cbe7d3689c616aaf402355356fde
Nov 04 01:45:19.852126 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:19.852103    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"]
Nov 04 01:45:20.614429 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:20.614400    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:20.692020 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:20.691983    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" event={"ID":"d2ae9460-8050-48a0-9632-7187c4a98fc5","Type":"ContainerStarted","Data":"3cfcd092902cf9f6e51c9d6729b16b1deb62cbe7d3689c616aaf402355356fde"}
Nov 04 01:45:20.861384 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:20.861359    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:21.120707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:21.120668    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:21.699798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:21.699764    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" event={"ID":"d2ae9460-8050-48a0-9632-7187c4a98fc5","Type":"ContainerStarted","Data":"c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f"}
Nov 04 01:45:21.700172 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:21.699830    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:21.817762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:21.817715    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" podStartSLOduration=2.894574131 podStartE2EDuration="3.817702163s" totalImagesPullingTime="923.128032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:21.816220836 +0000 UTC m=+652.532407703" watchObservedRunningTime="2026-11-04 01:45:21.817702163 +0000 UTC m=+652.533889019"
Nov 04 01:45:22.752301 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:22.752265    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:45:22.861279 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:22.861245    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:23.116935 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.116903    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:23.361968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.361931    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:23.503385 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.503308    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"]
Nov 04 01:45:23.506973 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.506957    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.610121 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.610092    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"]
Nov 04 01:45:23.615911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.615894    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-fa0149-serving-cert\""
Nov 04 01:45:23.647005 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.646979    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.647179 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.647034    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.647179 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.647111    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69btm\" (UniqueName: \"kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.647179 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.647139    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.747908 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.747875    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.748082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.747953    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.748082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.747980    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69btm\" (UniqueName: \"kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.748082 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:23.748052    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-serving-cert: secret "maas-api-e2e-worker-w3-fa0149-serving-cert" not found
Nov 04 01:45:23.748317 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.748088    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.748317 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:23.748140    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls podName:01ba092d-ff3f-4a1e-91d3-6b32095f129c nodeName:}" failed. No retries permitted until 2026-11-04 01:45:24.24811863 +0000 UTC m=+654.964305467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls") pod "maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" (UID: "01ba092d-ff3f-4a1e-91d3-6b32095f129c") : secret "maas-api-e2e-worker-w3-fa0149-serving-cert" not found
Nov 04 01:45:23.748708 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.748684    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.750796 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.750771    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.811053 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.810986    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69btm\" (UniqueName: \"kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:23.905163 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:23.905132    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:24.155015 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:24.154984    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:24.252434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:24.252398    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:24.254898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:24.254873    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\") pod \"maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:24.419654 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:24.419563    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:24.795691 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:24.795647    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"]
Nov 04 01:45:24.797870 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:45:24.797843    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ba092d_ff3f_4a1e_91d3_6b32095f129c.slice/crio-6cc24ec983b4d5203c2ec0ca7112b33085434511b114ad0cdb907569142d9396 WatchSource:0}: Error finding container 6cc24ec983b4d5203c2ec0ca7112b33085434511b114ad0cdb907569142d9396: Status 404 returned error can't find the container with id 6cc24ec983b4d5203c2ec0ca7112b33085434511b114ad0cdb907569142d9396
Nov 04 01:45:25.374138 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.374105    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:25.724412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.724102    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" event={"ID":"01ba092d-ff3f-4a1e-91d3-6b32095f129c","Type":"ContainerStarted","Data":"1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7"}
Nov 04 01:45:25.724412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.724162    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" event={"ID":"01ba092d-ff3f-4a1e-91d3-6b32095f129c","Type":"ContainerStarted","Data":"6cc24ec983b4d5203c2ec0ca7112b33085434511b114ad0cdb907569142d9396"}
Nov 04 01:45:25.724412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.724206    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:25.755197 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.755166    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:25.843749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.843691    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" podStartSLOduration=2.052964349 podStartE2EDuration="2.843672467s" totalImagesPullingTime="790.708118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:25.840997291 +0000 UTC m=+656.557184146" watchObservedRunningTime="2026-11-04 01:45:25.843672467 +0000 UTC m=+656.559859325"
Nov 04 01:45:25.995242 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:25.995150    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:27.710798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:27.710773    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:45:28.465276 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:28.465242    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:29.228716 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:29.228675    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:29.472500 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:29.472467    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:29.717302 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:29.717275    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:30.695247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:30.695219    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:45:31.734491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:31.734464    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:45:33.070951 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.070914    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:33.323378 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.323280    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:33.523859 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.523829    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"]
Nov 04 01:45:33.527686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.527670    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.628757 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.628734    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-795045\""
Nov 04 01:45:33.628907 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.628767    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-795045-dockercfg-l4plm\""
Nov 04 01:45:33.631756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.631734    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"]
Nov 04 01:45:33.640066 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.640046    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.640171 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.640083    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmcrx\" (UniqueName: \"kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx\") pod \"payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.740542 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.740513    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.740686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.740554    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmcrx\" (UniqueName: \"kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx\") pod \"payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.741230 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.741210    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.801022 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.800996    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmcrx\" (UniqueName: \"kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx\") pod \"payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:33.840080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:33.840049    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:34.212126 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.212099    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"]
Nov 04 01:45:34.214041 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:45:34.214011    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1fa8b9e_3fd1_44b9_8ede_e876217c93cf.slice/crio-5d06c7e7ca63a11061a26260f0acaaf53fb07a64ed31791e22876a23599e1717 WatchSource:0}: Error finding container 5d06c7e7ca63a11061a26260f0acaaf53fb07a64ed31791e22876a23599e1717: Status 404 returned error can't find the container with id 5d06c7e7ca63a11061a26260f0acaaf53fb07a64ed31791e22876a23599e1717
Nov 04 01:45:34.390361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.390314    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:34.636860 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.636822    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:34.771413 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.771379    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6" event={"ID":"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf","Type":"ContainerStarted","Data":"76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f"}
Nov 04 01:45:34.771587 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.771451    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6" event={"ID":"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf","Type":"ContainerStarted","Data":"5d06c7e7ca63a11061a26260f0acaaf53fb07a64ed31791e22876a23599e1717"}
Nov 04 01:45:34.771646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.771617    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:34.883380 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.883325    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:34.887618 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:34.887517    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6" podStartSLOduration=1.8875037799999999 podStartE2EDuration="1.88750378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:34.885862177 +0000 UTC m=+665.602049033" watchObservedRunningTime="2026-11-04 01:45:34.88750378 +0000 UTC m=+665.603690633"
Nov 04 01:45:39.839394 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:39.839301    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:44.488603 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.488544    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"]
Nov 04 01:45:44.493792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.493766    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.596978 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.596943    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-795045-serving-cert\""
Nov 04 01:45:44.599238 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.599215    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"]
Nov 04 01:45:44.640305 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.640268    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.640560 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.640370    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.640560 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.640398    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.640560 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.640436    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqzp\" (UniqueName: \"kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.741772 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.741693    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.741925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.741792    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.741925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.741809    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.741925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.741831    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqzp\" (UniqueName: \"kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.742511 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:44.742488    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-serving-cert: secret "maas-api-e2e-worker-w1-795045-serving-cert" not found
Nov 04 01:45:44.742631 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:44.742594    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls podName:8a60cfce-b37a-428a-bee3-bbe93eb9fb39 nodeName:}" failed. No retries permitted until 2026-11-04 01:45:45.242557903 +0000 UTC m=+675.958744738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls") pod "maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" (UID: "8a60cfce-b37a-428a-bee3-bbe93eb9fb39") : secret "maas-api-e2e-worker-w1-795045-serving-cert" not found
Nov 04 01:45:44.743182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.743159    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.744946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.744915    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:44.793854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.793810    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:44.805118 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:44.805088    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqzp\" (UniqueName: \"kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:45.042016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.041937    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:45.245303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.245268    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:45.248013 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.247988    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\") pod \"maas-api-e2e-worker-w1-795045-85f448c457-b9hz9\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:45.406829 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.406802    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:45.779448 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.779429    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"]
Nov 04 01:45:45.781055 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.781032    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:45:45.782205 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:45:45.782179    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a60cfce_b37a_428a_bee3_bbe93eb9fb39.slice/crio-4356d46883e2c3625e29b8c75c0ede671a09b5315e64db08f0f0a494f5b812fe WatchSource:0}: Error finding container 4356d46883e2c3625e29b8c75c0ede671a09b5315e64db08f0f0a494f5b812fe: Status 404 returned error can't find the container with id 4356d46883e2c3625e29b8c75c0ede671a09b5315e64db08f0f0a494f5b812fe
Nov 04 01:45:45.826294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:45.826267    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" event={"ID":"8a60cfce-b37a-428a-bee3-bbe93eb9fb39","Type":"ContainerStarted","Data":"4356d46883e2c3625e29b8c75c0ede671a09b5315e64db08f0f0a494f5b812fe"}
Nov 04 01:45:46.832304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:46.832273    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" event={"ID":"8a60cfce-b37a-428a-bee3-bbe93eb9fb39","Type":"ContainerStarted","Data":"1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3"}
Nov 04 01:45:46.832705 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:46.832437    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:46.950981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:46.950854    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" podStartSLOduration=2.14299165 podStartE2EDuration="2.950839902s" totalImagesPullingTime="807.848252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:46.948474142 +0000 UTC m=+677.664660998" watchObservedRunningTime="2026-11-04 01:45:46.950839902 +0000 UTC m=+677.667026755"
Nov 04 01:45:49.734775 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:49.734741    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:52.752176 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:45:52.752141    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:45:52.842233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:52.842206    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:45:54.566817 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:54.566783    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:57.711902 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:57.711870    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:58.344558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:58.344516    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:58.585919 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:58.585884    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:59.791277 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:45:59.791237    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:02.833343 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:02.833305    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:03.085105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:03.085025    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:03.652695 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:03.652665    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:03.907639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:03.907531    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:05.033919 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:05.033887    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:10.055990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:10.055934    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:10.304312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:10.304276    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:10.695714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:10.695682    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:10.957187 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:10.957094    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:12.952840 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:12.952801    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:13.564451 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:13.564420    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:13.855048 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:13.855015    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:15.561791 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:15.561760    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:20.239483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:20.239439    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:22.752058 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:46:22.752023    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:46:23.566587 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:23.566531    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:25.445890 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:25.445862    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:25.693185 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:25.693150    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:26.085523 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:26.085485    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:26.348838 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:26.348776    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:27.988552 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:27.988515    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:30.238841 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:30.238800    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:33.301409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:33.301359    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:35.039624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:35.039548    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:39.749220 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:39.749178    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:42.829215 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:42.829181    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:44.541070 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:44.541038    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:49.345324 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:49.345288    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:52.108306 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:52.108272    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:52.753243 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:46:52.753201    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:46:54.146145 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:54.146113    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:54.967743 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:54.967708    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:58.926965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:46:58.926922    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:03.652113 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:03.652076    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:08.469782 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:08.469752    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:13.250068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:13.249992    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:17.388926 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:17.388882    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:18.036197 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:18.036156    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:22.752034 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:47:22.751994    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:47:22.754352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:22.754320    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:23.910749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:23.910716    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:27.639360 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:27.639315    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:32.350088 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:32.350054    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:33.663148 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:33.663110    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:37.149508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:37.149469    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:38.316516 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:38.316478    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:41.943799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:41.943762    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:46.836788 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:46.836754    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:51.560533 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:51.560497    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:52.543964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.543933    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"]
Nov 04 01:47:52.547785 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.547762    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.583448 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.583426    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.583820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.583455    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.583820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.583479    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.583820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.583594    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.583820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.583633    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7vlh\" (UniqueName: \"kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.583820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.583664    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.649684 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.649659    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"kube-root-ca.crt\""
Nov 04 01:47:52.649684 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.649688    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"openshift-service-ca.crt\""
Nov 04 01:47:52.649863 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.649702    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"premium-w4-f4d202-kserve-self-signed-certs\""
Nov 04 01:47:52.650151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.650131    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"default-dockercfg-2hmgq\""
Nov 04 01:47:52.652136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.652120    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"]
Nov 04 01:47:52.685014 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.684985    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.685191 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.685029    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.685191 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.685060    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.685191 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.685088    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.685191 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.685137    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7vlh\" (UniqueName: \"kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.685191 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.685181    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.685466 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.685442    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.686112 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.686092    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.686224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.686203    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.687756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.687733    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.688942 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.688922    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.746551 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.746519    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:52.749521 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.749466    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7vlh\" (UniqueName: \"kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh\") pod \"premium-w4-f4d202-kserve-59dd7d87b4-5cj92\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") " pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:52.757326 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:47:52.757298    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:47:52.858643 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:52.858615    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:47:53.252042 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:53.252011    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"]
Nov 04 01:47:53.254323 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:47:53.254293    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff6c92e8_b5f6_46be_878d_848d15db27b4.slice/crio-1b384983a2a0e093e222c8f63d7246c6ce4eadc416d5022ac4d58683212d403a WatchSource:0}: Error finding container 1b384983a2a0e093e222c8f63d7246c6ce4eadc416d5022ac4d58683212d403a: Status 404 returned error can't find the container with id 1b384983a2a0e093e222c8f63d7246c6ce4eadc416d5022ac4d58683212d403a
Nov 04 01:47:53.502530 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:53.502445    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92" event={"ID":"ff6c92e8-b5f6-46be-878d-848d15db27b4","Type":"ContainerStarted","Data":"1b384983a2a0e093e222c8f63d7246c6ce4eadc416d5022ac4d58683212d403a"}
Nov 04 01:47:56.341852 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:56.341820    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:56.521596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:56.521544    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92" event={"ID":"ff6c92e8-b5f6-46be-878d-848d15db27b4","Type":"ContainerStarted","Data":"dca9f78da7f8f7d16b2a40cf188d20c3c1a2917fed2a550d2bcfb1326f4aded5"}
Nov 04 01:47:56.639228 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:56.639125    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92" podStartSLOduration=2.452921295 podStartE2EDuration="4.639112316s" totalImagesPullingTime="2.186191021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:47:56.637163546 +0000 UTC m=+807.353350402" watchObservedRunningTime="2026-11-04 01:47:56.639112316 +0000 UTC m=+807.355299171"
Nov 04 01:47:56.859384 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:47:56.859343    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:48:00.814066 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:00.814036    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:01.159717 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:01.159677    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:02.858755 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:02.858722    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:48:02.870636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:02.870611    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:48:03.570410 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:03.570379    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:48:05.936865 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:05.936825    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:10.835273 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:10.835238    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:15.642753 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:15.642720    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:20.447335 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:20.447295    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:22.752699 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:48:22.752663    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:48:24.254040 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:24.253997    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:25.244898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:25.244855    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:30.179727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:30.179685    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:34.908673 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:34.908634    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:39.950952 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:39.950879    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:44.859465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:44.859424    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:49.650347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:49.650310    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:52.750523 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:48:52.750488    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:48:54.206313 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.206282    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"]
Nov 04 01:48:54.209856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.209840    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.266591 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.265005    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"openshift-service-ca.crt\""
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.266834    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79cpk\" (UniqueName: \"kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.266891    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.266936    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"kube-root-ca.crt\""
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.266944    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.267028    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.267120    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.270062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.267164    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.314401 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.314371    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"distinct-w3-fa0149-kserve-self-signed-certs\""
Nov 04 01:48:54.314540 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.314403    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"default-dockercfg-9f9nf\""
Nov 04 01:48:54.316370 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.316352    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"]
Nov 04 01:48:54.368503 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.368465    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79cpk\" (UniqueName: \"kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.368683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.368494    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.368683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.368547    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.368683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.368589    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.368683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.368634    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.368683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.368674    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.369188 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.369141    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.369188 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.369186    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.369374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.369220    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.370937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.370912    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.371406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.371387    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.395976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.395949    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:54.441970 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.441941    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79cpk\" (UniqueName: \"kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk\") pod \"distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.521863 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.521783    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:54.725822 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.725783    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:54.945162 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:54.945084    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"]
Nov 04 01:48:55.863218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:55.863173    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb" event={"ID":"0d94dd6b-1341-4e63-bac9-60739173f300","Type":"ContainerStarted","Data":"1023eefe3fd0ad6ffb5fa8b08802c6528a74f7a6f6b822332d31110ad25c41df"}
Nov 04 01:48:55.863218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:55.863218    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb" event={"ID":"0d94dd6b-1341-4e63-bac9-60739173f300","Type":"ContainerStarted","Data":"35a4a14f8b433aadcaa7365a9d3caa3f18664d26b049bdbdfa8e6b4cc356ea03"}
Nov 04 01:48:55.981067 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:55.980996    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb" podStartSLOduration=1.9809794040000002 podStartE2EDuration="1.980979404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:48:55.98031725 +0000 UTC m=+866.696504106" watchObservedRunningTime="2026-11-04 01:48:55.980979404 +0000 UTC m=+866.697166261"
Nov 04 01:48:56.522379 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:56.522348    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:48:59.263138 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:48:59.263102    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:04.087833 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:04.087798    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:04.522786 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:04.522749    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:49:04.535901 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:04.535876    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:49:04.921840 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:04.921802    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:49:08.947968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:08.947934    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:13.958061 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:13.958019    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:18.765488 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:18.765460    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:21.014710 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.014679    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7117-3e8e8b"]
Nov 04 01:49:21.018259 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.018246    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:21.044309 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.044286    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtc6c\" (UniqueName: \"kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c\") pod \"test-curl-7117-3e8e8b\" (UID: \"6cfdae3e-9e98-419c-80ee-a125a27e9363\") " pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:21.073250 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.073225    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jw55s\""
Nov 04 01:49:21.076091 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.076069    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7117-3e8e8b"]
Nov 04 01:49:21.144745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.144720    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtc6c\" (UniqueName: \"kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c\") pod \"test-curl-7117-3e8e8b\" (UID: \"6cfdae3e-9e98-419c-80ee-a125a27e9363\") " pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:21.205517 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.205495    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtc6c\" (UniqueName: \"kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c\") pod \"test-curl-7117-3e8e8b\" (UID: \"6cfdae3e-9e98-419c-80ee-a125a27e9363\") " pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:21.331413 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.331323    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:21.704159 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:21.704138    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7117-3e8e8b"]
Nov 04 01:49:21.706432 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:49:21.706408    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cfdae3e_9e98_419c_80ee_a125a27e9363.slice/crio-800029615d289d6a3f31cfd139783ca99cdede57b61f6096338d9c0338789cfc WatchSource:0}: Error finding container 800029615d289d6a3f31cfd139783ca99cdede57b61f6096338d9c0338789cfc: Status 404 returned error can't find the container with id 800029615d289d6a3f31cfd139783ca99cdede57b61f6096338d9c0338789cfc
Nov 04 01:49:22.008920 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:22.008847    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7117-3e8e8b" event={"ID":"6cfdae3e-9e98-419c-80ee-a125a27e9363","Type":"ContainerStarted","Data":"800029615d289d6a3f31cfd139783ca99cdede57b61f6096338d9c0338789cfc"}
Nov 04 01:49:22.751368 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:22.751321    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:49:23.536340 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:23.536310    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:24.611403 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:24.611364    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:25.031646 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:25.031615    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7117-3e8e8b" event={"ID":"6cfdae3e-9e98-419c-80ee-a125a27e9363","Type":"ContainerStarted","Data":"03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176"}
Nov 04 01:49:25.145334 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:25.145290    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7117-3e8e8b" podStartSLOduration=2.468785235 podStartE2EDuration="5.145278841s" totalImagesPullingTime="2.676493606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:49:25.1440176 +0000 UTC m=+895.860204456" watchObservedRunningTime="2026-11-04 01:49:25.145278841 +0000 UTC m=+895.861465698"
Nov 04 01:49:26.618076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.618048    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7117-3e8e8b"]
Nov 04 01:49:26.622323 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.622296    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7117-3e8e8b"]
Nov 04 01:49:26.795582 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.795542    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7117-92f403"]
Nov 04 01:49:26.795988 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.795975    5039 cpu_manager.go:414] "containerMap: removing container" podUID="6cfdae3e-9e98-419c-80ee-a125a27e9363" containerName="test-curl-7117-3e8e8b"
Nov 04 01:49:26.796038 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.795990    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cfdae3e-9e98-419c-80ee-a125a27e9363" containerName="test-curl-7117-3e8e8b"
Nov 04 01:49:26.796072 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.796068    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cfdae3e-9e98-419c-80ee-a125a27e9363" containerName="test-curl-7117-3e8e8b"
Nov 04 01:49:26.799555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.799536    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:26.855862 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.855842    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7117-92f403"]
Nov 04 01:49:26.894807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.894745    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fdtq\" (UniqueName: \"kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq\") pod \"test-curl-7117-92f403\" (UID: \"db75edee-72fe-46bc-a47f-d7b24fc88e9f\") " pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:26.995489 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:26.995465    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fdtq\" (UniqueName: \"kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq\") pod \"test-curl-7117-92f403\" (UID: \"db75edee-72fe-46bc-a47f-d7b24fc88e9f\") " pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:27.040833 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:27.040806    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7117-3e8e8b" podUID="6cfdae3e-9e98-419c-80ee-a125a27e9363" containerName="test-curl-7117-3e8e8b" containerID="cri-o://03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176" gracePeriod=2
Nov 04 01:49:27.055864 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:27.055838    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fdtq\" (UniqueName: \"kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq\") pod \"test-curl-7117-92f403\" (UID: \"db75edee-72fe-46bc-a47f-d7b24fc88e9f\") " pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:27.091601 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:27.091555    5039 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7117-3e8e8b\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="6cfdae3e-9e98-419c-80ee-a125a27e9363" pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:27.111848 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:27.111825    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:27.480680 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:27.480529    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7117-92f403"]
Nov 04 01:49:27.482918 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:49:27.482888    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb75edee_72fe_46bc_a47f_d7b24fc88e9f.slice/crio-b77ecc903725ad18eaf18810ba4e25eb899e994c6210fa2729010851c174030d WatchSource:0}: Error finding container b77ecc903725ad18eaf18810ba4e25eb899e994c6210fa2729010851c174030d: Status 404 returned error can't find the container with id b77ecc903725ad18eaf18810ba4e25eb899e994c6210fa2729010851c174030d
Nov 04 01:49:28.047632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:28.047598    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7117-92f403" event={"ID":"db75edee-72fe-46bc-a47f-d7b24fc88e9f","Type":"ContainerStarted","Data":"7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a"}
Nov 04 01:49:28.048099 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:28.047645    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7117-92f403" event={"ID":"db75edee-72fe-46bc-a47f-d7b24fc88e9f","Type":"ContainerStarted","Data":"b77ecc903725ad18eaf18810ba4e25eb899e994c6210fa2729010851c174030d"}
Nov 04 01:49:28.160651 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:28.160605    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7117-92f403" podStartSLOduration=2.160591778 podStartE2EDuration="2.160591778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:49:28.159381984 +0000 UTC m=+898.875568837" watchObservedRunningTime="2026-11-04 01:49:28.160591778 +0000 UTC m=+898.876778627"
Nov 04 01:49:28.335407 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:28.335329    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:29.284529 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.284509    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:29.319598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.319500    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c\" (UniqueName: \"kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c\") pod \"6cfdae3e-9e98-419c-80ee-a125a27e9363\" (UID: \"6cfdae3e-9e98-419c-80ee-a125a27e9363\") "
Nov 04 01:49:29.321577 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.321542    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c" pod "6cfdae3e-9e98-419c-80ee-a125a27e9363" (UID: "6cfdae3e-9e98-419c-80ee-a125a27e9363"). InnerVolumeSpecName "kube-api-access-rtc6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:49:29.405499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.405473    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cfdae3e-9e98-419c-80ee-a125a27e9363" path="/var/lib/kubelet/pods/6cfdae3e-9e98-419c-80ee-a125a27e9363/volumes"
Nov 04 01:49:29.420598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.420561    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtc6c\" (UniqueName: \"kubernetes.io/projected/6cfdae3e-9e98-419c-80ee-a125a27e9363-kube-api-access-rtc6c\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:29.487537 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.487508    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:49:29.491252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.491230    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:49:29.743458 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.743419    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7117-92f403"]
Nov 04 01:49:29.749498 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:29.749463    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7117-92f403"]
Nov 04 01:49:30.060448 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:30.060351    5039 generic.go:356] "Generic (PLEG): container finished" podID="6cfdae3e-9e98-419c-80ee-a125a27e9363" containerID="03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176" exitCode=137
Nov 04 01:49:30.060624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:30.060443    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-3e8e8b"
Nov 04 01:49:30.060624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:30.060494    5039 scope.go:122] "RemoveContainer" containerID="03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176"
Nov 04 01:49:30.060804 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:30.060757    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7117-92f403" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" containerName="test-curl-7117-92f403" containerID="cri-o://7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a" gracePeriod=2
Nov 04 01:49:30.069326 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:30.069311    5039 scope.go:122] "RemoveContainer" containerID="03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176"
Nov 04 01:49:30.069621 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:30.069604    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176\": container with ID starting with 03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176 not found: ID does not exist" containerID="03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176"
Nov 04 01:49:30.069674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:30.069634    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176"} err="failed to get container status \"03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176\": rpc error: code = NotFound desc = could not find container \"03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176\": container with ID starting with 03224b750ac6349f92eaa8f34b8de4e477b4ae4f45c79bb8ba8a53faebbb4176 not found: ID does not exist"
Nov 04 01:49:30.112528 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:30.112502    5039 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7117-92f403\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:30.163727 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:30.163696    5039 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7117-92f403\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:31.934639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:31.934594    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:32.311588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:32.311550    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:32.349116 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:32.349087    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq\" (UniqueName: \"kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq\") pod \"db75edee-72fe-46bc-a47f-d7b24fc88e9f\" (UID: \"db75edee-72fe-46bc-a47f-d7b24fc88e9f\") "
Nov 04 01:49:32.351354 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:32.351316    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq" pod "db75edee-72fe-46bc-a47f-d7b24fc88e9f" (UID: "db75edee-72fe-46bc-a47f-d7b24fc88e9f"). InnerVolumeSpecName "kube-api-access-2fdtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:49:32.451003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:32.450969    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fdtq\" (UniqueName: \"kubernetes.io/projected/db75edee-72fe-46bc-a47f-d7b24fc88e9f-kube-api-access-2fdtq\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:32.594074 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:32.594043    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:32.852173 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:32.852090    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:33.081312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.081264    5039 generic.go:356] "Generic (PLEG): container finished" podID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" containerID="7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a" exitCode=137
Nov 04 01:49:33.081794 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.081327    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:33.081794 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.081411    5039 scope.go:122] "RemoveContainer" containerID="7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a"
Nov 04 01:49:33.092898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.092868    5039 scope.go:122] "RemoveContainer" containerID="7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a"
Nov 04 01:49:33.093171 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:33.093152    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a\": container with ID starting with 7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a not found: ID does not exist" containerID="7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a"
Nov 04 01:49:33.093251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.093179    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a"} err="failed to get container status \"7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a\": rpc error: code = NotFound desc = could not find container \"7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a\": container with ID starting with 7ad7ada4db6471f680e1a3194abc98d4c3336a5d1b9843f128888fe4bb117a3a not found: ID does not exist"
Nov 04 01:49:33.143091 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:33.142997    5039 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7117-92f403\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" pod="opendatahub/test-curl-7117-92f403"
Nov 04 01:49:33.162154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.162118    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:33.406071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:33.405995    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" path="/var/lib/kubelet/pods/db75edee-72fe-46bc-a47f-d7b24fc88e9f/volumes"
Nov 04 01:49:37.033983 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:37.033951    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:37.289795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:37.289712    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:37.823367 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:37.823326    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:38.392652 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:38.392609    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:40.294728 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.294694    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:40.539470 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.539436    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:40.581262 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.581195    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"]
Nov 04 01:49:40.581679 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.581665    5039 cpu_manager.go:414] "containerMap: removing container" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" containerName="test-curl-7117-92f403"
Nov 04 01:49:40.581733 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.581681    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" containerName="test-curl-7117-92f403"
Nov 04 01:49:40.581776 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.581766    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="db75edee-72fe-46bc-a47f-d7b24fc88e9f" containerName="test-curl-7117-92f403"
Nov 04 01:49:40.586266 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.586248    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.642969 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.642946    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"]
Nov 04 01:49:40.690090 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.690052    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-c62113-dockercfg-p4npc\""
Nov 04 01:49:40.690244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.690092    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-c62113\""
Nov 04 01:49:40.737035 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.737000    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krfqh\" (UniqueName: \"kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh\") pod \"payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.737173 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.737103    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.739470 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.739445    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"]
Nov 04 01:49:40.743176 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.743163    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:40.838512 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.838440    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:40.838512 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.838485    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.838720 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.838556    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk6k9\" (UniqueName: \"kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9\") pod \"payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:40.838758 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.838720    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krfqh\" (UniqueName: \"kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh\") pod \"payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.839142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.839125    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.866453 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.866427    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"]
Nov 04 01:49:40.902229 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.902208    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krfqh\" (UniqueName: \"kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh\") pod \"payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:40.939257 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.939231    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:40.939401 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.939291    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk6k9\" (UniqueName: \"kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9\") pod \"payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:40.940020 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.939995    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:40.999818 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:40.999796    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk6k9\" (UniqueName: \"kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9\") pod \"payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:41.056281 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:41.056257    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:41.199684 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:41.199651    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:41.432008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:41.431985    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"]
Nov 04 01:49:41.434015 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:49:41.433991    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21560ae3_d894_4d5d_9e16_bbbff6c3c215.slice/crio-fa24e9bc1873f81f41d7e304ccf1285379a7403b2d68ed259c0d9871db8a18c3 WatchSource:0}: Error finding container fa24e9bc1873f81f41d7e304ccf1285379a7403b2d68ed259c0d9871db8a18c3: Status 404 returned error can't find the container with id fa24e9bc1873f81f41d7e304ccf1285379a7403b2d68ed259c0d9871db8a18c3
Nov 04 01:49:41.573799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:41.573613    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"]
Nov 04 01:49:41.577313 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:49:41.577108    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ddc37a_dfea_4e44_b622_d9be7c3dba5a.slice/crio-85be47d827cd8cdf11ac27ee55e2d52702c16b09b2623eb71130b722f0731a22 WatchSource:0}: Error finding container 85be47d827cd8cdf11ac27ee55e2d52702c16b09b2623eb71130b722f0731a22: Status 404 returned error can't find the container with id 85be47d827cd8cdf11ac27ee55e2d52702c16b09b2623eb71130b722f0731a22
Nov 04 01:49:42.143553 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.143519    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt" event={"ID":"21560ae3-d894-4d5d-9e16-bbbff6c3c215","Type":"ContainerStarted","Data":"fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0"}
Nov 04 01:49:42.143762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.143597    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:42.143762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.143624    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt" event={"ID":"21560ae3-d894-4d5d-9e16-bbbff6c3c215","Type":"ContainerStarted","Data":"fa24e9bc1873f81f41d7e304ccf1285379a7403b2d68ed259c0d9871db8a18c3"}
Nov 04 01:49:42.145375 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.145343    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz" event={"ID":"15ddc37a-dfea-4e44-b622-d9be7c3dba5a","Type":"ContainerStarted","Data":"0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe"}
Nov 04 01:49:42.145522 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.145395    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz" event={"ID":"15ddc37a-dfea-4e44-b622-d9be7c3dba5a","Type":"ContainerStarted","Data":"85be47d827cd8cdf11ac27ee55e2d52702c16b09b2623eb71130b722f0731a22"}
Nov 04 01:49:42.145522 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.145422    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:42.259487 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.259433    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt" podStartSLOduration=2.259416537 podStartE2EDuration="2.259416537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:49:42.258147916 +0000 UTC m=+912.974334774" watchObservedRunningTime="2026-11-04 01:49:42.259416537 +0000 UTC m=+912.975603395"
Nov 04 01:49:42.374298 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:42.374242    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz" podStartSLOduration=2.37422528 podStartE2EDuration="2.37422528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:49:42.372298169 +0000 UTC m=+913.088485023" watchObservedRunningTime="2026-11-04 01:49:42.37422528 +0000 UTC m=+913.090412137"
Nov 04 01:49:43.360653 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:43.360618    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:48.147689 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:48.147656    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:52.752129 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:49:52.752095    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:49:52.947673 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:52.947624    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:53.155830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:53.155802    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:49:53.156008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:53.155884    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:49:57.747986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:49:57.747951    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:02.551139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:02.551104    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:06.763334 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:06.763297    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:06.980672 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:06.980635    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"]
Nov 04 01:50:06.999052 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:06.999020    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.043750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.043684    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"]
Nov 04 01:50:07.083129 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.083101    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzsrd\" (UniqueName: \"kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.083252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.083136    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.083252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.083165    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.083365 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.083338    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.090510 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.090491    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-c62113-serving-cert\""
Nov 04 01:50:07.184254 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.184223    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzsrd\" (UniqueName: \"kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.184418 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.184267    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.184418 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.184299    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.184418 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.184383    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.184418 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:50:07.184388    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-serving-cert: secret "maas-api-e2e-extmod-a-c62113-serving-cert" not found
Nov 04 01:50:07.184678 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:50:07.184462    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls podName:6ed265b4-85d7-4663-89c5-d7a404871a75 nodeName:}" failed. No retries permitted until 2026-11-04 01:50:07.684441894 +0000 UTC m=+938.400628728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls") pod "maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" (UID: "6ed265b4-85d7-4663-89c5-d7a404871a75") : secret "maas-api-e2e-extmod-a-c62113-serving-cert" not found
Nov 04 01:50:07.185223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.185102    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.186938 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.186918    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.245110 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.245072    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzsrd\" (UniqueName: \"kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.300899 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.300796    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:07.547788 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.547755    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:07.688520 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.688493    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.691094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.691070    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:07.913263 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:07.913229    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:08.288979 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:08.288957    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"]
Nov 04 01:50:08.291731 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:50:08.291704    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed265b4_85d7_4663_89c5_d7a404871a75.slice/crio-1d0c77df0e942148a69555e86974ce3dec67f08ac461d874597606ab7a4c0724 WatchSource:0}: Error finding container 1d0c77df0e942148a69555e86974ce3dec67f08ac461d874597606ab7a4c0724: Status 404 returned error can't find the container with id 1d0c77df0e942148a69555e86974ce3dec67f08ac461d874597606ab7a4c0724
Nov 04 01:50:09.291461 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:09.291432    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" event={"ID":"6ed265b4-85d7-4663-89c5-d7a404871a75","Type":"ContainerStarted","Data":"1d0c77df0e942148a69555e86974ce3dec67f08ac461d874597606ab7a4c0724"}
Nov 04 01:50:10.299595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:10.299454    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" event={"ID":"6ed265b4-85d7-4663-89c5-d7a404871a75","Type":"ContainerStarted","Data":"62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97"}
Nov 04 01:50:10.299595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:10.299550    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:10.418477 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:10.418431    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" podStartSLOduration=3.450475281 podStartE2EDuration="4.418418585s" totalImagesPullingTime="967.943304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:10.415700584 +0000 UTC m=+941.131887438" watchObservedRunningTime="2026-11-04 01:50:10.418418585 +0000 UTC m=+941.134605440"
Nov 04 01:50:12.146965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:12.146922    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:15.554308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:15.554274    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:16.322649 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:16.322619    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:50:16.965898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:16.965838    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:21.734554 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:21.734512    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:22.751372 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:50:22.751323    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:50:24.960519 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:24.960481    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:26.521612 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:26.521555    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:31.341555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:31.341514    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:32.053263 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:32.053220    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:32.298965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:32.298934    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:36.143326 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:36.143291    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:38.567419 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.567391    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"]
Nov 04 01:50:38.572686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.572661    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.654526 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.654499    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"default-dockercfg-r4744\""
Nov 04 01:50:38.704150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.704123    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"distinct-w2-82706f-kserve-self-signed-certs\""
Nov 04 01:50:38.704150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.704165    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"kube-root-ca.crt\""
Nov 04 01:50:38.704347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.704208    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"openshift-service-ca.crt\""
Nov 04 01:50:38.719716 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.719694    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.719837 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.719745    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.719837 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.719768    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbvx9\" (UniqueName: \"kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.719917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.719837    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.719917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.719907    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.719989 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.719931    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.735704 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.735683    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"]
Nov 04 01:50:38.821462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821377    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821414    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvx9\" (UniqueName: \"kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821437    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821481    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821513    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821563    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821908 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821867    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.821961 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.821915    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.822118 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.822093    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.824089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.824062    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.824174 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.824159    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.829489 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.829465    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:38.883199 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.883167    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbvx9\" (UniqueName: \"kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9\") pod \"distinct-w2-82706f-kserve-6598fd7fbb-l5ggh\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:38.883857 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:38.883825    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:39.278420 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:39.278392    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"]
Nov 04 01:50:39.280020 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:50:39.279996    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2300fa57_0a7a_46a5_8e9e_52eb879d6c67.slice/crio-1612dd4ab9869ca778b581734b882993b54c923fc32dd6ead96069ff4f098720 WatchSource:0}: Error finding container 1612dd4ab9869ca778b581734b882993b54c923fc32dd6ead96069ff4f098720: Status 404 returned error can't find the container with id 1612dd4ab9869ca778b581734b882993b54c923fc32dd6ead96069ff4f098720
Nov 04 01:50:39.479982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:39.479944    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh" event={"ID":"2300fa57-0a7a-46a5-8e9e-52eb879d6c67","Type":"ContainerStarted","Data":"cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2"}
Nov 04 01:50:39.479982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:39.479987    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh" event={"ID":"2300fa57-0a7a-46a5-8e9e-52eb879d6c67","Type":"ContainerStarted","Data":"1612dd4ab9869ca778b581734b882993b54c923fc32dd6ead96069ff4f098720"}
Nov 04 01:50:39.596469 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:39.596414    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh" podStartSLOduration=1.596397018 podStartE2EDuration="1.596397018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:39.595551211 +0000 UTC m=+970.311738067" watchObservedRunningTime="2026-11-04 01:50:39.596397018 +0000 UTC m=+970.312583872"
Nov 04 01:50:39.884676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:39.884602    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:41.148613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:41.148582    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:42.010359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:42.010327    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:42.627750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:42.627719    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:42.882860 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:42.882779    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:45.855547 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:45.855512    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:47.461556 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:47.461527    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:47.713455 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:47.713366    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:48.457222 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:48.457181    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:48.884127 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:48.884095    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:48.897743 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:48.897720    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:49.553775 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:49.553741    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:50:49.869146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:49.869112    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:50.116921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:50.116887    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:50.508548 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:50.508504    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:50.749754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:50.749717    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:51.321406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.321377    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:51.576365 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.576294    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:51.718559 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.718528    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"]
Nov 04 01:50:51.722439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.722417    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:51.774550 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.774503    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-06099f\""
Nov 04 01:50:51.822866 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.822831    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:51.823722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.823698    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-06099f-dockercfg-bq8gw\""
Nov 04 01:50:51.827368 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.827280    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"]
Nov 04 01:50:51.849126 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.849092    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28n88\" (UniqueName: \"kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88\") pod \"payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:51.849271 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.849222    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:51.877067 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.877040    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"]
Nov 04 01:50:51.880743 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.880729    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:51.939615 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.939588    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"]
Nov 04 01:50:51.950103 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.950074    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:51.950236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.950211    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pqpd\" (UniqueName: \"kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd\") pod \"payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:51.950344 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.950322    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:51.950408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.950385    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28n88\" (UniqueName: \"kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88\") pod \"payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:51.950884 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:51.950858    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:52.011111 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.011082    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28n88\" (UniqueName: \"kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88\") pod \"payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:52.041076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.041050    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:52.051321 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.051296    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pqpd\" (UniqueName: \"kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd\") pod \"payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:52.051464 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.051339    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:52.052295 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.052258    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:52.114007 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.113947    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pqpd\" (UniqueName: \"kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd\") pod \"payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:52.193710 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.193677    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:52.422679 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.422052    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"]
Nov 04 01:50:52.560209 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.560163    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" event={"ID":"97e32fc6-86f0-4652-b427-5a993fb5b6bf","Type":"ContainerStarted","Data":"aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5"}
Nov 04 01:50:52.560402 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.560384    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:50:52.560465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.560421    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" event={"ID":"97e32fc6-86f0-4652-b427-5a993fb5b6bf","Type":"ContainerStarted","Data":"b9e659a402fd8d03119318a3026b10ff3c94d29ae181f0336e01e9eeaf09dab0"}
Nov 04 01:50:52.572188 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.572166    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"]
Nov 04 01:50:52.574728 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:50:52.574702    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bef7c76_970c_463d_9eb8_88c042ddee45.slice/crio-302e43e7c445b10c03d55ef9b761bce23b1e213491756855e2ff695e937a03b2 WatchSource:0}: Error finding container 302e43e7c445b10c03d55ef9b761bce23b1e213491756855e2ff695e937a03b2: Status 404 returned error can't find the container with id 302e43e7c445b10c03d55ef9b761bce23b1e213491756855e2ff695e937a03b2
Nov 04 01:50:52.577649 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.577629    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"]
Nov 04 01:50:52.577854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.577809    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy" containerID="cri-o://8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32" gracePeriod=30
Nov 04 01:50:52.685793 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:52.685704    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" podStartSLOduration=1.685690164 podStartE2EDuration="1.685690164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:52.683988486 +0000 UTC m=+983.400175343" watchObservedRunningTime="2026-11-04 01:50:52.685690164 +0000 UTC m=+983.401877020"
Nov 04 01:50:52.755129 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:50:52.755090    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:50:53.567370 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:53.567320    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" event={"ID":"9bef7c76-970c-463d-9eb8-88c042ddee45","Type":"ContainerStarted","Data":"a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45"}
Nov 04 01:50:53.567843 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:53.567387    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" event={"ID":"9bef7c76-970c-463d-9eb8-88c042ddee45","Type":"ContainerStarted","Data":"302e43e7c445b10c03d55ef9b761bce23b1e213491756855e2ff695e937a03b2"}
Nov 04 01:50:53.567843 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:53.567501    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:50:53.683903 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:53.683859    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" podStartSLOduration=2.68384675 podStartE2EDuration="2.68384675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:53.681915989 +0000 UTC m=+984.398102845" watchObservedRunningTime="2026-11-04 01:50:53.68384675 +0000 UTC m=+984.400033652"
Nov 04 01:50:55.259031 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:55.259001    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:55.512143 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:55.512063    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:56.518439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:56.518384    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:57.843215 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.843191    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:50:57.908252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908226    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908303    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908362    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908411    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908460    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908514    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908545    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908633    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908669    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs\") pod \"e100a836-e762-49ab-89c2-b82c0af447e9\" (UID: \"e100a836-e762-49ab-89c2-b82c0af447e9\") "
Nov 04 01:50:57.908869 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.908815    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:50:57.909361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.909067    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:50:57.909361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.909316    5039 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:57.909361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.909338    5039 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e100a836-e762-49ab-89c2-b82c0af447e9-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:57.909746 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.909709    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:50:57.909864 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.909843    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:50:57.910048 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.910008    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:50:57.911776 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.911746    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:50:57.911872 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.911815    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "kube-api-access-cvzqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:50:57.911943 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.911919    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:50:57.912025 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:57.911987    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo" pod "e100a836-e762-49ab-89c2-b82c0af447e9" (UID: "e100a836-e762-49ab-89c2-b82c0af447e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:50:58.009955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.009919    5039 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.009955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.009953    5039 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.010142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.009968    5039 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e100a836-e762-49ab-89c2-b82c0af447e9-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.010142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.009981    5039 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.010142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.009994    5039 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e100a836-e762-49ab-89c2-b82c0af447e9-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.010142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.010008    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvzqj\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-kube-api-access-cvzqj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.010142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.010022    5039 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e100a836-e762-49ab-89c2-b82c0af447e9-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:50:58.514581 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.514536    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:58.598101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.598071    5039 generic.go:356] "Generic (PLEG): container finished" podID="e100a836-e762-49ab-89c2-b82c0af447e9" containerID="8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32" exitCode=0
Nov 04 01:50:58.598284 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.598132    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"
Nov 04 01:50:58.598284 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.598145    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" event={"ID":"e100a836-e762-49ab-89c2-b82c0af447e9","Type":"ContainerDied","Data":"8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32"}
Nov 04 01:50:58.598284 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.598204    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd" event={"ID":"e100a836-e762-49ab-89c2-b82c0af447e9","Type":"ContainerDied","Data":"b47e36b7334a9880fb3596ebe8a685e6c32a5e7d80ef401d3daa95fc486a17e6"}
Nov 04 01:50:58.598284 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.598228    5039 scope.go:122] "RemoveContainer" containerID="8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32"
Nov 04 01:50:58.608054 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.608040    5039 scope.go:122] "RemoveContainer" containerID="8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32"
Nov 04 01:50:58.608304 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:50:58.608285    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32\": container with ID starting with 8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32 not found: ID does not exist" containerID="8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32"
Nov 04 01:50:58.608361 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.608318    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32"} err="failed to get container status \"8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32\": rpc error: code = NotFound desc = could not find container \"8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32\": container with ID starting with 8a55e652fc98d14cd29e338afa14cb30cabfcf26e701d6527a6f29915a65ac32 not found: ID does not exist"
Nov 04 01:50:58.763282 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.763257    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"]
Nov 04 01:50:58.768810 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.768710    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:58.769128 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.769106    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd"]
Nov 04 01:50:58.828117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.828094    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"]
Nov 04 01:50:58.828637 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.828623    5039 cpu_manager.go:414] "containerMap: removing container" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy"
Nov 04 01:50:58.828691 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.828639    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy"
Nov 04 01:50:58.828738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.828729    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" containerName="istio-proxy"
Nov 04 01:50:58.833173 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.833158    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:58.887899 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.887873    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-c2cde5\""
Nov 04 01:50:58.887899 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.887877    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-c2cde5-dockercfg-b9qf2\""
Nov 04 01:50:58.890967 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.890946    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"]
Nov 04 01:50:58.918931 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.918907    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:58.919063 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.918958    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwms6\" (UniqueName: \"kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6\") pod \"payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:58.974543 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.974513    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"]
Nov 04 01:50:58.978222 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:58.978203    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.020468 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.020405    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:59.020468 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.020450    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwms6\" (UniqueName: \"kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6\") pod \"payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:59.021063 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.021043    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:59.036642 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.036618    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"]
Nov 04 01:50:59.083553 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.083526    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwms6\" (UniqueName: \"kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6\") pod \"payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:59.121556 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.121527    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.121695 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.121668    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkrz6\" (UniqueName: \"kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6\") pod \"payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.146444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.146421    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:59.224810 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.223322    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkrz6\" (UniqueName: \"kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6\") pod \"payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.224810 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.223504    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.224810 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.224479    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.284309 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.284236    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkrz6\" (UniqueName: \"kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6\") pod \"payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.290882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.290855    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:50:59.408264 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.408229    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e100a836-e762-49ab-89c2-b82c0af447e9" path="/var/lib/kubelet/pods/e100a836-e762-49ab-89c2-b82c0af447e9/volumes"
Nov 04 01:50:59.526367 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.526342    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"]
Nov 04 01:50:59.527755 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:50:59.527725    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7ac2bd_8e28_4401_82e2_bef16d73fdfe.slice/crio-19316a0e7033a4d4112504704fc16461ea5459860b82875741763628861e2fb0 WatchSource:0}: Error finding container 19316a0e7033a4d4112504704fc16461ea5459860b82875741763628861e2fb0: Status 404 returned error can't find the container with id 19316a0e7033a4d4112504704fc16461ea5459860b82875741763628861e2fb0
Nov 04 01:50:59.607549 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.607524    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc" event={"ID":"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe","Type":"ContainerStarted","Data":"526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b"}
Nov 04 01:50:59.607681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.607618    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc" event={"ID":"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe","Type":"ContainerStarted","Data":"19316a0e7033a4d4112504704fc16461ea5459860b82875741763628861e2fb0"}
Nov 04 01:50:59.607681 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.607659    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:50:59.669982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.669963    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"]
Nov 04 01:50:59.671546 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:50:59.671517    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5edd0f_d73d_42f6_be0a_4772d180b196.slice/crio-2fa79510172d01ffecc44261e071b316fe15add79b11ca81fce94e1b7312a3e3 WatchSource:0}: Error finding container 2fa79510172d01ffecc44261e071b316fe15add79b11ca81fce94e1b7312a3e3: Status 404 returned error can't find the container with id 2fa79510172d01ffecc44261e071b316fe15add79b11ca81fce94e1b7312a3e3
Nov 04 01:50:59.725098 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:50:59.725041    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc" podStartSLOduration=1.725023819 podStartE2EDuration="1.725023819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:59.721558937 +0000 UTC m=+990.437745787" watchObservedRunningTime="2026-11-04 01:50:59.725023819 +0000 UTC m=+990.441210670"
Nov 04 01:51:00.615373 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:00.615335    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf" event={"ID":"9b5edd0f-d73d-42f6-be0a-4772d180b196","Type":"ContainerStarted","Data":"e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2"}
Nov 04 01:51:00.615782 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:00.615425    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf" event={"ID":"9b5edd0f-d73d-42f6-be0a-4772d180b196","Type":"ContainerStarted","Data":"2fa79510172d01ffecc44261e071b316fe15add79b11ca81fce94e1b7312a3e3"}
Nov 04 01:51:00.615782 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:00.615667    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:51:00.732387 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:00.732331    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf" podStartSLOduration=2.732317258 podStartE2EDuration="2.732317258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:00.72991025 +0000 UTC m=+991.446097123" watchObservedRunningTime="2026-11-04 01:51:00.732317258 +0000 UTC m=+991.448504148"
Nov 04 01:51:01.444364 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:01.444324    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:03.570625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:03.570593    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:51:04.577840 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:04.577808    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:51:06.155886 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:06.155850    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:10.179003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.178971    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"]
Nov 04 01:51:10.182804 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.182784    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.234888 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.234826    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-82706f-openshift-default-dockercfg-v862d\""
Nov 04 01:51:10.287269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.287240    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"]
Nov 04 01:51:10.329176 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329142    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329357 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329265    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ht4n\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329357 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329303    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329467 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329397    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329467 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329433    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329511    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329656 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329641    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329705 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329690    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.329772 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.329756    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430589 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430511    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430589 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430542    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430589 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430585    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430628    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430663    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ht4n\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430692    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430723    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430751    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.430830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.430794    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.431123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.431055    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.431123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.431101    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.431445 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.431419    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.431592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.431428    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.431913 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.431888    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.433474 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.433451    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.433562 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.433487    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.491087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.491064    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.491267 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.491247    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ht4n\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.496965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.496945    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:10.618931 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.618903    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:51:10.927462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.927432    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"]
Nov 04 01:51:10.929038 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:51:10.929013    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd87e327_135e_4a72_9925_4d28d4a9360a.slice/crio-f0f749f58621bcc35dc7d55dc5b2fecf52ecf7fd4a924a39f8a4ba53ba63989e WatchSource:0}: Error finding container f0f749f58621bcc35dc7d55dc5b2fecf52ecf7fd4a924a39f8a4ba53ba63989e: Status 404 returned error can't find the container with id f0f749f58621bcc35dc7d55dc5b2fecf52ecf7fd4a924a39f8a4ba53ba63989e
Nov 04 01:51:10.931604 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.931581    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:10.931688 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.931592    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:10.931688 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.931670    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:10.931754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:10.931704    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:11.178749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:11.178668    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:11.645594 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:11.644762    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:51:11.677715 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:11.677679    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" event={"ID":"bd87e327-135e-4a72-9925-4d28d4a9360a","Type":"ContainerStarted","Data":"3f26adaa8045ece0db8d30f637144146c2330ae042c6390508e221474aa2f109"}
Nov 04 01:51:11.677873 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:11.677747    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" event={"ID":"bd87e327-135e-4a72-9925-4d28d4a9360a","Type":"ContainerStarted","Data":"f0f749f58621bcc35dc7d55dc5b2fecf52ecf7fd4a924a39f8a4ba53ba63989e"}
Nov 04 01:51:11.879082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:11.879033    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podStartSLOduration=1.879018845 podStartE2EDuration="1.879018845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:11.877857081 +0000 UTC m=+1002.594043932" watchObservedRunningTime="2026-11-04 01:51:11.879018845 +0000 UTC m=+1002.595205701"
Nov 04 01:51:12.497286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:12.497248    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:12.499113 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:12.499085    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 01:51:12.499233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:12.499142    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 01:51:13.498094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:13.498061    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 01:51:13.498498 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:13.498126    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 01:51:14.497457 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:14.497419    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 01:51:14.497653 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:14.497476    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 01:51:15.497808 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:15.497776    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 01:51:15.498218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:15.497828    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 01:51:15.793067 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:15.792970    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:16.497434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:16.497400    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 04 01:51:16.497638 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:16.497461    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 04 01:51:17.790468 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:17.790424    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:17.791320 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:17.791285    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:17.791969 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:17.791953    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:51:20.202314 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.202273    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"]
Nov 04 01:51:20.206368 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.206347    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.222981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.222959    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.223101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.223014    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2jmh\" (UniqueName: \"kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.223101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.223038    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.223228 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.223207    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.258583 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.258552    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-06099f-serving-cert\""
Nov 04 01:51:20.310095 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.310063    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"]
Nov 04 01:51:20.324108 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.324073    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.324241 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.324163    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2jmh\" (UniqueName: \"kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.324241 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:51:20.324203    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-serving-cert: secret "maas-api-e2e-extmod-a-06099f-serving-cert" not found
Nov 04 01:51:20.324350 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:51:20.324267    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls podName:a962fc40-e1d7-4fc4-bf4c-700a108f15cd nodeName:}" failed. No retries permitted until 2026-11-04 01:51:20.824250916 +0000 UTC m=+1011.540437754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls") pod "maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" (UID: "a962fc40-e1d7-4fc4-bf4c-700a108f15cd") : secret "maas-api-e2e-extmod-a-06099f-serving-cert" not found
Nov 04 01:51:20.324350 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.324203    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.324520 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.324412    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.324990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.324973    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.326882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.326859    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.386795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.386764    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2jmh\" (UniqueName: \"kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.498719 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.498642    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:20.781322 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.781232    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:20.828897 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.828872    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:20.831443 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:20.831410    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:21.119338 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:21.119310    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:21.495444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:21.495420    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"]
Nov 04 01:51:21.497738 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:51:21.497710    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda962fc40_e1d7_4fc4_bf4c_700a108f15cd.slice/crio-e4cce39c46cdcfddd845881edf46a6e6fdd831daa006f59132dd75969a4e76df WatchSource:0}: Error finding container e4cce39c46cdcfddd845881edf46a6e6fdd831daa006f59132dd75969a4e76df: Status 404 returned error can't find the container with id e4cce39c46cdcfddd845881edf46a6e6fdd831daa006f59132dd75969a4e76df
Nov 04 01:51:21.499331 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:21.499316    5039 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:51:21.738408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:21.738378    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" event={"ID":"a962fc40-e1d7-4fc4-bf4c-700a108f15cd","Type":"ContainerStarted","Data":"e4cce39c46cdcfddd845881edf46a6e6fdd831daa006f59132dd75969a4e76df"}
Nov 04 01:51:22.748612 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:22.748530    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" event={"ID":"a962fc40-e1d7-4fc4-bf4c-700a108f15cd","Type":"ContainerStarted","Data":"3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572"}
Nov 04 01:51:22.749050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:22.748736    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:22.757304 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:51:22.757275    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:51:22.865882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:22.865826    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" podStartSLOduration=1.865650188 podStartE2EDuration="2.865808921s" totalImagesPullingTime="1.000158733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:22.864243478 +0000 UTC m=+1013.580430336" watchObservedRunningTime="2026-11-04 01:51:22.865808921 +0000 UTC m=+1013.581995778"
Nov 04 01:51:25.305885 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:25.305850    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:25.550949 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:25.550914    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:27.742814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:27.742760    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:28.760769 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:28.760741    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:51:30.277803 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:30.277758    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:34.621958 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.621920    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"]
Nov 04 01:51:34.627234 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.627216    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.679111 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.679084    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-c2cde5-serving-cert\""
Nov 04 01:51:34.730777 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.730751    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"]
Nov 04 01:51:34.768144 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.768104    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75z88\" (UniqueName: \"kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.768430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.768408    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.768709 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.768693    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.768847 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.768834    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.869621 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.869551    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.869818 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.869631    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.869818 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.869698    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75z88\" (UniqueName: \"kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.869818 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.869749    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.870192 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.870170    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.870297 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:51:34.870262    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-serving-cert: secret "maas-api-e2e-extmod-b-c2cde5-serving-cert" not found
Nov 04 01:51:34.870427 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:51:34.870358    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls podName:f1cbe519-06cc-42da-abd0-f96acc05dc61 nodeName:}" failed. No retries permitted until 2026-11-04 01:51:35.370337139 +0000 UTC m=+1026.086523990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls") pod "maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" (UID: "f1cbe519-06cc-42da-abd0-f96acc05dc61") : secret "maas-api-e2e-extmod-b-c2cde5-serving-cert" not found
Nov 04 01:51:34.872863 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.872804    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:34.927256 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.927224    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:34.936090 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:34.936060    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75z88\" (UniqueName: \"kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:35.176685 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:35.176600    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:35.375761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:35.375728    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:35.378138 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:35.378107    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\") pod \"maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:35.540153 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:35.540074    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:35.922864 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:35.922840    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"]
Nov 04 01:51:35.925518 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:51:35.925484    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cbe519_06cc_42da_abd0_f96acc05dc61.slice/crio-af48457be52bf641a460619f04cfb5ea313170ff5045d3aeba63b0ba1dfaf463 WatchSource:0}: Error finding container af48457be52bf641a460619f04cfb5ea313170ff5045d3aeba63b0ba1dfaf463: Status 404 returned error can't find the container with id af48457be52bf641a460619f04cfb5ea313170ff5045d3aeba63b0ba1dfaf463
Nov 04 01:51:36.838330 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:36.838236    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" event={"ID":"f1cbe519-06cc-42da-abd0-f96acc05dc61","Type":"ContainerStarted","Data":"0c69e8809846d9fc016471547560e28dccfce2d7721802198d2b780984e61599"}
Nov 04 01:51:36.838330 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:36.838298    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" event={"ID":"f1cbe519-06cc-42da-abd0-f96acc05dc61","Type":"ContainerStarted","Data":"af48457be52bf641a460619f04cfb5ea313170ff5045d3aeba63b0ba1dfaf463"}
Nov 04 01:51:36.975562 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:36.975502    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" podStartSLOduration=2.165575333 podStartE2EDuration="2.975489881s" totalImagesPullingTime="809.914548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:36.973497139 +0000 UTC m=+1027.689683995" watchObservedRunningTime="2026-11-04 01:51:36.975489881 +0000 UTC m=+1027.691676736"
Nov 04 01:51:37.742909 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:37.742859    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:37.847016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:37.846940    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:39.840466 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:39.840240    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:41.077385 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.077352    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"]
Nov 04 01:51:41.081274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.081255    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.133342 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.133320    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-f4d202-openshift-default-dockercfg-6vjrk\""
Nov 04 01:51:41.235876 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.235847    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236042 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.235889    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236042 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.235910    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb8wf\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236042 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.235973    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236164 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.236060    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236164 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.236091    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236164 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.236111    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236164 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.236131    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.236300 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.236170    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.302752 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.302721    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"]
Nov 04 01:51:41.336836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.336760    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.336836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.336803    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.336836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.336821    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb8wf\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337114 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.336849    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337275 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337252    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337302    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337325    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337460 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337418    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337460 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337451    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337561 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337497    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337868 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337841    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.337963 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.337524    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.338115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.338091    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.338314 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.338291    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.340243 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.340217    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.340385 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.340361    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.399269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.399235    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.399374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.399364    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb8wf\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:41.694430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:41.694402    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:42.127191 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:42.127163    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"]
Nov 04 01:51:42.128949 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:51:42.128926    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42599b98_ce4c_461d_9cfa_b3f3d38d9153.slice/crio-687adbfc70a57d9b5a87b54f94b29edffefdac8e627f69bdc22d98c407ecd30e WatchSource:0}: Error finding container 687adbfc70a57d9b5a87b54f94b29edffefdac8e627f69bdc22d98c407ecd30e: Status 404 returned error can't find the container with id 687adbfc70a57d9b5a87b54f94b29edffefdac8e627f69bdc22d98c407ecd30e
Nov 04 01:51:42.131426 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:42.131387    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:42.131491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:42.131474    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:42.131531 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:42.131516    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:42.880854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:42.880822    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" event={"ID":"42599b98-ce4c-461d-9cfa-b3f3d38d9153","Type":"ContainerStarted","Data":"c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084"}
Nov 04 01:51:42.880854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:42.880872    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" event={"ID":"42599b98-ce4c-461d-9cfa-b3f3d38d9153","Type":"ContainerStarted","Data":"687adbfc70a57d9b5a87b54f94b29edffefdac8e627f69bdc22d98c407ecd30e"}
Nov 04 01:51:43.100008 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.099951    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podStartSLOduration=2.099935418 podStartE2EDuration="2.099935418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:43.049249271 +0000 UTC m=+1033.765436127" watchObservedRunningTime="2026-11-04 01:51:43.099935418 +0000 UTC m=+1033.816122274"
Nov 04 01:51:43.486592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.481461    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:43.694846 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.694806    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:43.696441 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.696414    5039 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:51:43.696593 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.696461    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:51:43.737643 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.736977    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:43.858946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:43.858917    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:51:44.695070 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:44.695036    5039 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:51:44.695482 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:44.695091    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:51:44.708265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:44.708235    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:45.695110 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:45.695079    5039 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:51:45.695541 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:45.695130    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:51:46.370270 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:46.370231    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:46.654333 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:46.654236    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:46.695749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:46.695719    5039 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:51:46.696125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:46.695774    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:51:47.695706 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:47.695676    5039 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:51:47.695875 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:47.695725    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:51:47.803118 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:47.803081    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:48.223224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:48.223166    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:48.695676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:48.695637    5039 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:51:48.695874 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:48.695695    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 04 01:51:49.660438 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:49.660401    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:49.700806 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:49.700777    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:49.701236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:49.701218    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:49.701987 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:49.701967    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:51:52.752048 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:51:52.752010    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:51:54.627402 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:54.627369    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:54.874278 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:54.874202    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:59.432866 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:51:59.432832    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:04.241294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:04.241263    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:09.137601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:09.137546    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:13.761767 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:13.761734    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:14.345912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:14.345859    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:15.142779 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:15.142747    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:15.736781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:15.736748    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:16.138952 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:16.138917    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:19.386173 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:19.386143    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:19.649683 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:19.649602    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:19.898487 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:19.898460    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:22.751212 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:52:22.751177    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:52:24.575898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:24.575863    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:25.655195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.655164    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:25.777484 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.777448    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"]
Nov 04 01:52:25.781939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.781919    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.831853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.831830    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.831981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.831920    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.831981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.831942    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.832058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.832033    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.832105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.832081    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.832139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.832107    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.832182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.832139    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4j5b\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.832217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.832198    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.832261 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.832244    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933493 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933420    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933493 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933473    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933493 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933491    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933534    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933606    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933632    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933647    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4j5b\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933673    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.933731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.933711    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.934103 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.934030    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.934103 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.934054    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.934103 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.934100    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.934332 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.934150    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.934555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.934538    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.935929 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.935907    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.936386 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.936370    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.990226 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.990202    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"]
Nov 04 01:52:25.997889 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.997865    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:25.999801 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:25.999359    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4j5b\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:26.096727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:26.096705    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:26.527503 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:26.527480    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"]
Nov 04 01:52:26.529216 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:26.529190    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52880f5a_85cd_403f_95e1_c1f08a3bb9bc.slice/crio-d6026890224d35d80217e2381c4bd429941f3f647bf9f5b36a5bfaaa65cda95a WatchSource:0}: Error finding container d6026890224d35d80217e2381c4bd429941f3f647bf9f5b36a5bfaaa65cda95a: Status 404 returned error can't find the container with id d6026890224d35d80217e2381c4bd429941f3f647bf9f5b36a5bfaaa65cda95a
Nov 04 01:52:26.531816 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:26.531785    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:26.531918 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:26.531848    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:26.531918 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:26.531874    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:27.095752 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:27.095714    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:27.168768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:27.168729    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" event={"ID":"52880f5a-85cd-403f-95e1-c1f08a3bb9bc","Type":"ContainerStarted","Data":"4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15"}
Nov 04 01:52:27.168958 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:27.168815    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" event={"ID":"52880f5a-85cd-403f-95e1-c1f08a3bb9bc","Type":"ContainerStarted","Data":"d6026890224d35d80217e2381c4bd429941f3f647bf9f5b36a5bfaaa65cda95a"}
Nov 04 01:52:27.374125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:27.374007    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podStartSLOduration=2.373988441 podStartE2EDuration="2.373988441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:27.338029611 +0000 UTC m=+1078.054216467" watchObservedRunningTime="2026-11-04 01:52:27.373988441 +0000 UTC m=+1078.090175298"
Nov 04 01:52:27.684169 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:27.684082    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:27.968276 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:27.968195    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.014297 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.014262    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"]
Nov 04 01:52:28.018628 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.018607    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.071583 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.071529    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-c0ab5f\""
Nov 04 01:52:28.071758 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.071608    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-c0ab5f-dockercfg-cfx72\""
Nov 04 01:52:28.078870 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.078846    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"]
Nov 04 01:52:28.097751 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.097731    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:28.099234 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.099209    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:52:28.099784 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.099453    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 04 01:52:28.166595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.163308    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9f7x\" (UniqueName: \"kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x\") pod \"payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.166595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.163368    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.166595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.165205    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"]
Nov 04 01:52:28.169224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.169200    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.210325 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.210234    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"]
Nov 04 01:52:28.259044 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.258966    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.264759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.264180    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45kc5\" (UniqueName: \"kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5\") pod \"payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.264759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.264332    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9f7x\" (UniqueName: \"kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x\") pod \"payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.264759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.264612    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.265032 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.264942    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.266127 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.266102    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.354893 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.354324    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9f7x\" (UniqueName: \"kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x\") pod \"payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.366188 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.366143    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.366352 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.366269    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45kc5\" (UniqueName: \"kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5\") pod \"payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.366882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.366856    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.439590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.437197    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45kc5\" (UniqueName: \"kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5\") pod \"payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.490136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.490107    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:28.542644 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.525066    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.652685 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.652340    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:28.855792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.855509    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.898680 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:28.898650    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"]
Nov 04 01:52:28.900633 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:28.900607    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87635f6_06e0_44fe_9aed_49dd0f2d5511.slice/crio-87d99280d8102d3f66e55ff60e0bdad6fcf2eeb913041950a121df07dafa420d WatchSource:0}: Error finding container 87d99280d8102d3f66e55ff60e0bdad6fcf2eeb913041950a121df07dafa420d: Status 404 returned error can't find the container with id 87d99280d8102d3f66e55ff60e0bdad6fcf2eeb913041950a121df07dafa420d
Nov 04 01:52:29.051256 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.049390    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"]
Nov 04 01:52:29.053476 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:29.053450    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ebcda0_06c7_4aed_94b1_00389a56b35d.slice/crio-51c1a5891d972c6d2525ce4a260eec5110c92bf1c1683b57b08644c37bda65b2 WatchSource:0}: Error finding container 51c1a5891d972c6d2525ce4a260eec5110c92bf1c1683b57b08644c37bda65b2: Status 404 returned error can't find the container with id 51c1a5891d972c6d2525ce4a260eec5110c92bf1c1683b57b08644c37bda65b2
Nov 04 01:52:29.097759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.097736    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:52:29.098067 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.097786    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 04 01:52:29.111324 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.111301    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:29.188539 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.188500    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l" event={"ID":"b87635f6-06e0-44fe-9aed-49dd0f2d5511","Type":"ContainerStarted","Data":"95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a"}
Nov 04 01:52:29.188762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.188583    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l" event={"ID":"b87635f6-06e0-44fe-9aed-49dd0f2d5511","Type":"ContainerStarted","Data":"87d99280d8102d3f66e55ff60e0bdad6fcf2eeb913041950a121df07dafa420d"}
Nov 04 01:52:29.188762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.188614    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:29.190379 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.190335    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" event={"ID":"b6ebcda0-06c7-4aed-94b1-00389a56b35d","Type":"ContainerStarted","Data":"9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06"}
Nov 04 01:52:29.190507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.190396    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" event={"ID":"b6ebcda0-06c7-4aed-94b1-00389a56b35d","Type":"ContainerStarted","Data":"51c1a5891d972c6d2525ce4a260eec5110c92bf1c1683b57b08644c37bda65b2"}
Nov 04 01:52:29.190595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.190543    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:29.310089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.310039    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l" podStartSLOduration=1.310025201 podStartE2EDuration="1.310025201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:29.306754432 +0000 UTC m=+1080.022941287" watchObservedRunningTime="2026-11-04 01:52:29.310025201 +0000 UTC m=+1080.026212056"
Nov 04 01:52:29.442648 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:29.436607    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" podStartSLOduration=2.436589002 podStartE2EDuration="2.436589002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:29.431931295 +0000 UTC m=+1080.148118154" watchObservedRunningTime="2026-11-04 01:52:29.436589002 +0000 UTC m=+1080.152775861"
Nov 04 01:52:30.043593 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:30.042796    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:30.097501 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:30.097441    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:52:30.097702 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:30.097513    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 04 01:52:30.453167 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:30.453108    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:30.698261 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:30.698230    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:31.067036 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:31.066988    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:31.097128 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:31.097097    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:52:31.097280 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:31.097150    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 04 01:52:31.335644 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:31.331997    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:32.097145 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:32.097109    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:52:32.097651 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:32.097164    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 04 01:52:32.313447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:32.313416    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:33.097418 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:33.097383    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:52:33.097916 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:33.097441    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 04 01:52:34.102028 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:34.101998    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:34.102427 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:34.102380    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:34.103155 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:34.103131    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:52:34.847882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:34.847848    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:37.187223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:37.187183    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"]
Nov 04 01:52:37.431777 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:37.431740    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:38.499139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.499106    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"]
Nov 04 01:52:38.504179 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.504162    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585002 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.584970    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585043    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chnv5\" (UniqueName: \"kubernetes.io/projected/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-kube-api-access-chnv5\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585231 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585204    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585280 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585245    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585280 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585271    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585350 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585338    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585403 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585369    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585403 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585393    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.585505 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.585435    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686552 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686507    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686591    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686617    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686662    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686694    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686746    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chnv5\" (UniqueName: \"kubernetes.io/projected/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-kube-api-access-chnv5\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686861    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686892    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.686955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686915    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.687139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.686977    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.687139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.687026    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.687139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.687039    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.687265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.687198    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.687431 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.687404    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.689101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.689075    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.689223 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.689162    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.720063 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.720029    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"]
Nov 04 01:52:38.747091 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.747068    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chnv5\" (UniqueName: \"kubernetes.io/projected/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-kube-api-access-chnv5\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.747401 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.747378    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qpptj\" (UID: \"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:38.817745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:38.817684    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:39.276243 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.276211    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"]
Nov 04 01:52:39.278006 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:39.277982    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb8bdd3_cd7f_40d0_8886_48ea8c759cdc.slice/crio-42ee3abb5337dc519b0ee370cd0731213a5ab602022a183e40122203981e7ef1 WatchSource:0}: Error finding container 42ee3abb5337dc519b0ee370cd0731213a5ab602022a183e40122203981e7ef1: Status 404 returned error can't find the container with id 42ee3abb5337dc519b0ee370cd0731213a5ab602022a183e40122203981e7ef1
Nov 04 01:52:39.280091 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.280061    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.280154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.280124    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.280190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.280173    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.629100 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.629065    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:39.699397 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.699369    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"]
Nov 04 01:52:39.703799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.703777    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.757825 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.757799    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 01:52:39.757986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.757881    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 01:52:39.757986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.757919    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 01:52:39.757986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.757960    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-5kbxt\""
Nov 04 01:52:39.758437 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.758414    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-754dba0eabb0\""
Nov 04 01:52:39.797935 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.797900    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.798125 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.797953    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.798197 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.798136    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.798282 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.798253    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c57jq\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.807860 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.807834    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"]
Nov 04 01:52:39.856138 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.856108    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 01:52:39.898888 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.898821    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c57jq\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.898888 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.898871    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.899054 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.898900    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.899054 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.898972    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.899536 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.899510    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.899704 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.899686    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.901425 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.901409    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:39.959309 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:39.959276    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c57jq\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq\") pod \"payload-processing-external-model-8ffbc55d8-rqd7j\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") " pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:40.017260 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.017230    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:40.070250 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.070188    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:40.199558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.199483    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:52:40.199732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.199696    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:52:40.265732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.265155    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" event={"ID":"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc","Type":"ContainerStarted","Data":"f741229a2e79e92a8bf9dd92a0b567feb13d9881e4f1ce8d4d2f2b786a0eea7a"}
Nov 04 01:52:40.265732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.265216    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" event={"ID":"dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc","Type":"ContainerStarted","Data":"42ee3abb5337dc519b0ee370cd0731213a5ab602022a183e40122203981e7ef1"}
Nov 04 01:52:40.460057 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.457950    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podStartSLOduration=2.457929462 podStartE2EDuration="2.457929462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:40.45489595 +0000 UTC m=+1091.171082809" watchObservedRunningTime="2026-11-04 01:52:40.457929462 +0000 UTC m=+1091.174116319"
Nov 04 01:52:40.488911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.488877    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"]
Nov 04 01:52:40.500488 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:40.500458    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3fb56a_6e1e_4569_9d88_6bcc688e8322.slice/crio-35cd0539f5af0e74585bda79d509d74e475493faa4c524ccd25676f8c09dfc5b WatchSource:0}: Error finding container 35cd0539f5af0e74585bda79d509d74e475493faa4c524ccd25676f8c09dfc5b: Status 404 returned error can't find the container with id 35cd0539f5af0e74585bda79d509d74e475493faa4c524ccd25676f8c09dfc5b
Nov 04 01:52:40.710946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.709440    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:40.818390 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.818348    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:40.820080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.820055    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:40.820210 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.820103    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:40.978225 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:40.978116    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:41.275991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:41.275877    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" event={"ID":"6a3fb56a-6e1e-4569-9d88-6bcc688e8322","Type":"ContainerStarted","Data":"6d622152d7fdc07a61966624917e8036da7fd48513fc1db5495ab03b1b9a1372"}
Nov 04 01:52:41.276265 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:41.276249    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:52:41.276369 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:41.276294    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" event={"ID":"6a3fb56a-6e1e-4569-9d88-6bcc688e8322","Type":"ContainerStarted","Data":"35cd0539f5af0e74585bda79d509d74e475493faa4c524ccd25676f8c09dfc5b"}
Nov 04 01:52:41.397050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:41.396998    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podStartSLOduration=2.396982538 podStartE2EDuration="2.396982538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:41.394447546 +0000 UTC m=+1092.110634403" watchObservedRunningTime="2026-11-04 01:52:41.396982538 +0000 UTC m=+1092.113169393"
Nov 04 01:52:41.818688 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:41.818649    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:41.819146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:41.818733    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:42.789740 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:42.789680    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:42.818134 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:42.818104    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:42.818296 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:42.818182    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:43.574244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.574214    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:43.818792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.818717    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:43.818792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.818776    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:43.824654 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.822536    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"]
Nov 04 01:52:43.846588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.835799    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:43.884423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.884393    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"]
Nov 04 01:52:43.954818 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.949118    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c2cde5\"/\"payload-processing-external-model-plugins-e2e-extmod-b-c2cde5\""
Nov 04 01:52:43.959185 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.959151    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c2cde5\"/\"payload-processing-external-model-e2e--596b314e-dockercfg-dkcqg\""
Nov 04 01:52:43.959682 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.959662    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c2cde5\"/\"routing-overlay\""
Nov 04 01:52:43.960014 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.959994    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c2cde5\"/\"agc-cred-4f24e0a360fd\""
Nov 04 01:52:43.960661 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.960644    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c2cde5\"/\"kube-root-ca.crt\""
Nov 04 01:52:43.960761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.960749    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c2cde5\"/\"openshift-service-ca.crt\""
Nov 04 01:52:43.976930 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.976897    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:43.977078 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.976974    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:43.977078 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.977061    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:43.977182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:43.977121    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv49s\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.077911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.077824    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.077911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.077879    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.077911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.077901    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.078195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.077931    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv49s\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.078586 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.078546    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.078740 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.078721    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.080780 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.080757    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.167592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.149758    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv49s\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s\") pod \"payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") " pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.169595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.167806    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:44.581591 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.581540    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"]
Nov 04 01:52:44.584405 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:44.584371    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713734bc_821d_4844_9073_d60237efbdf7.slice/crio-bdef0960c2de227ac11faf52078f47b804949da149ea4ec6599e1f23ba5c52ec WatchSource:0}: Error finding container bdef0960c2de227ac11faf52078f47b804949da149ea4ec6599e1f23ba5c52ec: Status 404 returned error can't find the container with id bdef0960c2de227ac11faf52078f47b804949da149ea4ec6599e1f23ba5c52ec
Nov 04 01:52:44.768359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.768327    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:44.818805 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.818776    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:44.818980 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:44.818823    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:45.040326 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.040293    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:45.285728 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.285689    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:45.310080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.309351    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm" event={"ID":"713734bc-821d-4844-9073-d60237efbdf7","Type":"ContainerStarted","Data":"3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1"}
Nov 04 01:52:45.310080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.309414    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm" event={"ID":"713734bc-821d-4844-9073-d60237efbdf7","Type":"ContainerStarted","Data":"bdef0960c2de227ac11faf52078f47b804949da149ea4ec6599e1f23ba5c52ec"}
Nov 04 01:52:45.310080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.309521    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:45.442937 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.442851    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm" podStartSLOduration=2.442833031 podStartE2EDuration="2.442833031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:45.438339533 +0000 UTC m=+1096.154526391" watchObservedRunningTime="2026-11-04 01:52:45.442833031 +0000 UTC m=+1096.159019890"
Nov 04 01:52:45.820367 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.820333    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:45.820920 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:45.820393    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:46.776596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:46.775721    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vkm88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:52:46.776596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:46.775793    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88" podUID="dbeaf941-77d4-45f8-8e94-825323dc4a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:52:46.818660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:46.818624    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qpptj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:52:46.818851 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:46.818690    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj" podUID="dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 01:52:48.455242 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:48.455211    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:48.457259 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:48.457236    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:48.461698 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:48.461672    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj"
Nov 04 01:52:49.844840 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:49.844806    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:50.090338 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:50.090293    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:50.773836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:50.773780    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:51.043734 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.043613    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:51.091341 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.091311    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"]
Nov 04 01:52:51.095965 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.095938    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.198686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.198649    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-79badd-dockercfg-6mmv6\""
Nov 04 01:52:51.198686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.198684    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-79badd\""
Nov 04 01:52:51.203144 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.203111    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"]
Nov 04 01:52:51.256269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.253183    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"]
Nov 04 01:52:51.271420 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.271390    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.285950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.285919    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttnq8\" (UniqueName: \"kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8\") pod \"payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.286106 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.286022    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.307719 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.307634    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"]
Nov 04 01:52:51.387883 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.387847    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttnq8\" (UniqueName: \"kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8\") pod \"payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.388083 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.387886    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.388083 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.388076    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.388217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.388107    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2mlz\" (UniqueName: \"kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz\") pod \"payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.388905 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.388878    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.450727 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.450698    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttnq8\" (UniqueName: \"kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8\") pod \"payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.488976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.488940    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2mlz\" (UniqueName: \"kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz\") pod \"payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.489205 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.489186    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.489953 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.489935    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.549703 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.549674    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2mlz\" (UniqueName: \"kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz\") pod \"payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.588592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.588504    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:51.710717 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.710628    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:51.828910 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.828870    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"]
Nov 04 01:52:51.970098 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:51.970073    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"]
Nov 04 01:52:51.972520 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:51.972494    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116a331b_3176_4ecc_ae7a_e19f0d594911.slice/crio-15e56cc7f9cfeda1001031ecffc5b2e8653059fbaaf274d7814bbcfc5d0b8814 WatchSource:0}: Error finding container 15e56cc7f9cfeda1001031ecffc5b2e8653059fbaaf274d7814bbcfc5d0b8814: Status 404 returned error can't find the container with id 15e56cc7f9cfeda1001031ecffc5b2e8653059fbaaf274d7814bbcfc5d0b8814
Nov 04 01:52:52.095903 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.095871    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"]
Nov 04 01:52:52.099525 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:52.099499    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7add59_2961_4fbd_b42a_f8dc223ad08d.slice/crio-a2a4a94d78fd52c411764bb31de1f9f5b6efb7687505464db8caa13f1e6de367 WatchSource:0}: Error finding container a2a4a94d78fd52c411764bb31de1f9f5b6efb7687505464db8caa13f1e6de367: Status 404 returned error can't find the container with id a2a4a94d78fd52c411764bb31de1f9f5b6efb7687505464db8caa13f1e6de367
Nov 04 01:52:52.211036 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.211006    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92" podUID="ff6c92e8-b5f6-46be-878d-848d15db27b4" containerName="main" containerID="cri-o://dca9f78da7f8f7d16b2a40cf188d20c3c1a2917fed2a550d2bcfb1326f4aded5" gracePeriod=45
Nov 04 01:52:52.285956 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.285872    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:52:52.309753 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.309724    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:52.373464 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.373411    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl" event={"ID":"ac7add59-2961-4fbd-b42a-f8dc223ad08d","Type":"ContainerStarted","Data":"57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb"}
Nov 04 01:52:52.373694 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.373478    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:52:52.373694 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.373503    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl" event={"ID":"ac7add59-2961-4fbd-b42a-f8dc223ad08d","Type":"ContainerStarted","Data":"a2a4a94d78fd52c411764bb31de1f9f5b6efb7687505464db8caa13f1e6de367"}
Nov 04 01:52:52.375340 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.375312    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z" event={"ID":"116a331b-3176-4ecc-ae7a-e19f0d594911","Type":"ContainerStarted","Data":"415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378"}
Nov 04 01:52:52.375496 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.375368    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z" event={"ID":"116a331b-3176-4ecc-ae7a-e19f0d594911","Type":"ContainerStarted","Data":"15e56cc7f9cfeda1001031ecffc5b2e8653059fbaaf274d7814bbcfc5d0b8814"}
Nov 04 01:52:52.375496 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.375399    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:52:52.377364 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.377343    5039 generic.go:356] "Generic (PLEG): container finished" podID="ff6c92e8-b5f6-46be-878d-848d15db27b4" containerID="dca9f78da7f8f7d16b2a40cf188d20c3c1a2917fed2a550d2bcfb1326f4aded5" exitCode=0
Nov 04 01:52:52.377451 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.377397    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92" event={"ID":"ff6c92e8-b5f6-46be-878d-848d15db27b4","Type":"ContainerDied","Data":"dca9f78da7f8f7d16b2a40cf188d20c3c1a2917fed2a550d2bcfb1326f4aded5"}
Nov 04 01:52:52.489062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.489018    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl" podStartSLOduration=1.489005266 podStartE2EDuration="1.489005266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:52.487891889 +0000 UTC m=+1103.204078746" watchObservedRunningTime="2026-11-04 01:52:52.489005266 +0000 UTC m=+1103.205192122"
Nov 04 01:52:52.489602 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.489553    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:52:52.501687 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501665    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir\") pod \"ff6c92e8-b5f6-46be-878d-848d15db27b4\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") "
Nov 04 01:52:52.501798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501725    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home\") pod \"ff6c92e8-b5f6-46be-878d-848d15db27b4\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") "
Nov 04 01:52:52.501798 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501753    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache\") pod \"ff6c92e8-b5f6-46be-878d-848d15db27b4\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") "
Nov 04 01:52:52.501913 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501802    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm\") pod \"ff6c92e8-b5f6-46be-878d-848d15db27b4\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") "
Nov 04 01:52:52.501913 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501837    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh\" (UniqueName: \"kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh\") pod \"ff6c92e8-b5f6-46be-878d-848d15db27b4\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") "
Nov 04 01:52:52.502089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501865    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs\") pod \"ff6c92e8-b5f6-46be-878d-848d15db27b4\" (UID: \"ff6c92e8-b5f6-46be-878d-848d15db27b4\") "
Nov 04 01:52:52.502089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.501918    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir" pod "ff6c92e8-b5f6-46be-878d-848d15db27b4" (UID: "ff6c92e8-b5f6-46be-878d-848d15db27b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.502089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.502064    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home" pod "ff6c92e8-b5f6-46be-878d-848d15db27b4" (UID: "ff6c92e8-b5f6-46be-878d-848d15db27b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.502089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.502071    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache" pod "ff6c92e8-b5f6-46be-878d-848d15db27b4" (UID: "ff6c92e8-b5f6-46be-878d-848d15db27b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.502597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.502549    5039 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.502597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.502595    5039 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.503101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.502611    5039 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.504601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.504580    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs" pod "ff6c92e8-b5f6-46be-878d-848d15db27b4" (UID: "ff6c92e8-b5f6-46be-878d-848d15db27b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:52:52.504601 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.504582    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh" pod "ff6c92e8-b5f6-46be-878d-848d15db27b4" (UID: "ff6c92e8-b5f6-46be-878d-848d15db27b4"). InnerVolumeSpecName "kube-api-access-f7vlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:52.504695 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.504624    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm" pod "ff6c92e8-b5f6-46be-878d-848d15db27b4" (UID: "ff6c92e8-b5f6-46be-878d-848d15db27b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.555148 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.555086    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:52.603113 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.603083    5039 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff6c92e8-b5f6-46be-878d-848d15db27b4-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.603339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.603114    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7vlh\" (UniqueName: \"kubernetes.io/projected/ff6c92e8-b5f6-46be-878d-848d15db27b4-kube-api-access-f7vlh\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.603339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.603130    5039 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff6c92e8-b5f6-46be-878d-848d15db27b4-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.604929 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:52.604889    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z" podStartSLOduration=1.604876261 podStartE2EDuration="1.604876261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:52.603032168 +0000 UTC m=+1103.319219023" watchObservedRunningTime="2026-11-04 01:52:52.604876261 +0000 UTC m=+1103.321063117"
Nov 04 01:52:52.757284 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:52:52.757246    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:52:53.385793 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.385763    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"
Nov 04 01:52:53.386244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.385832    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92" event={"ID":"ff6c92e8-b5f6-46be-878d-848d15db27b4","Type":"ContainerDied","Data":"1b384983a2a0e093e222c8f63d7246c6ce4eadc416d5022ac4d58683212d403a"}
Nov 04 01:52:53.386244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.385878    5039 scope.go:122] "RemoveContainer" containerID="dca9f78da7f8f7d16b2a40cf188d20c3c1a2917fed2a550d2bcfb1326f4aded5"
Nov 04 01:52:53.549057 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.549024    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"]
Nov 04 01:52:53.554691 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.554669    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/premium-w4-f4d202-kserve-59dd7d87b4-5cj92"]
Nov 04 01:52:53.604991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.604956    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"]
Nov 04 01:52:53.605506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.605493    5039 cpu_manager.go:414] "containerMap: removing container" podUID="ff6c92e8-b5f6-46be-878d-848d15db27b4" containerName="main"
Nov 04 01:52:53.605563 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.605507    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff6c92e8-b5f6-46be-878d-848d15db27b4" containerName="main"
Nov 04 01:52:53.605769 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.605747    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff6c92e8-b5f6-46be-878d-848d15db27b4" containerName="main"
Nov 04 01:52:53.610818 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.610800    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.672397 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.672268    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"]
Nov 04 01:52:53.712035 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712003    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712053    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712147    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712299 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712190    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712360 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712286    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9xqb\" (UniqueName: \"kubernetes.io/projected/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-kube-api-access-j9xqb\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712360 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712343    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712472 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712410    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712550 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712535    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.712634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.712590    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.813723 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813692    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.813887 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813733    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.813887 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813757    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.813887 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813787    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.813887 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813826    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.813887 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813868    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813923    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9xqb\" (UniqueName: \"kubernetes.io/projected/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-kube-api-access-j9xqb\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813972    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814080 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.813997    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.814149    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.814327    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814506 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.814456    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.814985 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.814953    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.815116 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.815093    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.816631 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.816611    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.816707 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.816654    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.874706 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.874673    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.874871 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.874738    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9xqb\" (UniqueName: \"kubernetes.io/projected/e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f-kube-api-access-j9xqb\") pod \"maas-default-gateway-openshift-default-778b7b5c47-kdtlh\" (UID: \"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:53.927919 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:53.927826    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:54.385890 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:54.385766    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"]
Nov 04 01:52:54.388345 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:52:54.388314    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a9a5ce_74d6_479e_ba63_6c79f0e75a1f.slice/crio-7d9f345cf5369cbbea8fce290e31a1a19ba35c7a953e04be2d539b727ef4881c WatchSource:0}: Error finding container 7d9f345cf5369cbbea8fce290e31a1a19ba35c7a953e04be2d539b727ef4881c: Status 404 returned error can't find the container with id 7d9f345cf5369cbbea8fce290e31a1a19ba35c7a953e04be2d539b727ef4881c
Nov 04 01:52:54.390632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:54.390598    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:54.390730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:54.390678    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:54.390792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:54.390751    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:55.264615 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.264407    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:55.406464 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.406435    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff6c92e8-b5f6-46be-878d-848d15db27b4" path="/var/lib/kubelet/pods/ff6c92e8-b5f6-46be-878d-848d15db27b4/volumes"
Nov 04 01:52:55.406828 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.406761    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" event={"ID":"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f","Type":"ContainerStarted","Data":"5cde283ccab158c93e352b5082a26b0fbfcec2642ce597bbc3847f93f9036c64"}
Nov 04 01:52:55.406828 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.406806    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" event={"ID":"e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f","Type":"ContainerStarted","Data":"7d9f345cf5369cbbea8fce290e31a1a19ba35c7a953e04be2d539b727ef4881c"}
Nov 04 01:52:55.626594 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.625670    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podStartSLOduration=2.625651489 podStartE2EDuration="2.625651489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:55.572037149 +0000 UTC m=+1106.288224006" watchObservedRunningTime="2026-11-04 01:52:55.625651489 +0000 UTC m=+1106.341838347"
Nov 04 01:52:55.859961 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.859926    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:55.937452 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.930873    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:52:55.951590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.945682    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:52:55.951590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:55.945744    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:52:56.320423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:56.320349    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:52:56.840592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:56.835719    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" containerName="payload-processing" containerID="cri-o://df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d" gracePeriod=25
Nov 04 01:52:56.952595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:56.941845    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:52:56.952595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:56.941903    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:52:57.929318 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:57.929283    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:52:57.930087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:57.930058    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:52:58.933596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:58.932266    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:52:58.933596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:58.932323    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:52:59.945754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:59.945706    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:52:59.946251 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:52:59.945767    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:53:00.076312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:00.076275    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:00.694264 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:00.694229    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:00.928725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:00.928694    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:53:00.928890 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:00.928748    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:53:01.618168 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:01.618135    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:01.868088 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:01.868056    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:01.929050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:01.928967    5039 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-kdtlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 01:53:01.929050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:01.929031    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh" podUID="e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 01:53:02.284644 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:02.284532    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:03.207949 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:03.207919    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:53:03.208495 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:03.208321    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:53:03.209236 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:03.209219    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh"
Nov 04 01:53:03.389353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:03.389329    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:53:03.389498 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:03.389415    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:53:04.724631 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:04.724600    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:04.984478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:04.984398    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:09.671810 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:09.671771    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:10.693824 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:10.693783    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:12.284329 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:12.284296    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:14.443989 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:14.443952    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:14.916365 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:14.916324    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:15.218477 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:15.218390    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:15.531761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:15.531685    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:17.110915 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.110860    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:53:17.198732 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.198704    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9\" (UniqueName: \"kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9\") pod \"65b4910a-7de8-4924-b37c-e98b8b324615\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") "
Nov 04 01:53:17.198916 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.198833    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume\") pod \"65b4910a-7de8-4924-b37c-e98b8b324615\" (UID: \"65b4910a-7de8-4924-b37c-e98b8b324615\") "
Nov 04 01:53:17.199182 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.199160    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume" pod "65b4910a-7de8-4924-b37c-e98b8b324615" (UID: "65b4910a-7de8-4924-b37c-e98b8b324615"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:17.199260 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.199241    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65b4910a-7de8-4924-b37c-e98b8b324615-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:17.201109 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.201078    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9" pod "65b4910a-7de8-4924-b37c-e98b8b324615" (UID: "65b4910a-7de8-4924-b37c-e98b8b324615"). InnerVolumeSpecName "kube-api-access-jp9k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:17.300155 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.300125    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp9k9\" (UniqueName: \"kubernetes.io/projected/65b4910a-7de8-4924-b37c-e98b8b324615-kube-api-access-jp9k9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:17.557110 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.557075    5039 generic.go:356] "Generic (PLEG): container finished" podID="65b4910a-7de8-4924-b37c-e98b8b324615" containerID="df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d" exitCode=0
Nov 04 01:53:17.557110 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.557115    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"
Nov 04 01:53:17.557347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.557148    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" event={"ID":"65b4910a-7de8-4924-b37c-e98b8b324615","Type":"ContainerDied","Data":"df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d"}
Nov 04 01:53:17.557347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.557213    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6" event={"ID":"65b4910a-7de8-4924-b37c-e98b8b324615","Type":"ContainerDied","Data":"1c8d52c9d025afc70a948a9423d8bc3c08a1594c3958c3861a6d33c16e1dd357"}
Nov 04 01:53:17.557347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.557238    5039 scope.go:122] "RemoveContainer" containerID="df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d"
Nov 04 01:53:17.569444 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.569425    5039 scope.go:122] "RemoveContainer" containerID="df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d"
Nov 04 01:53:17.569766 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:53:17.569748    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d\": container with ID starting with df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d not found: ID does not exist" containerID="df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d"
Nov 04 01:53:17.569822 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.569772    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d"} err="failed to get container status \"df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d\": rpc error: code = NotFound desc = could not find container \"df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d\": container with ID starting with df23f6de5a868d1bccc22201872ec86aff195ae8d446f5c0209a100e980b4b8d not found: ID does not exist"
Nov 04 01:53:17.720290 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.720214    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"]
Nov 04 01:53:17.771376 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:17.771335    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6"]
Nov 04 01:53:19.231177 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:19.231134    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:19.408625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:19.408590    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" path="/var/lib/kubelet/pods/65b4910a-7de8-4924-b37c-e98b8b324615/volumes"
Nov 04 01:53:21.227054 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:21.226997    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:21.482042 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:21.481964    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:22.283688 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:22.283653    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:22.756167 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:53:22.756131    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:53:23.492171 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:23.492133    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:25.238162 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:25.238126    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:25.990355 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:25.990323    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:26.273049 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:26.272973    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:27.664170 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:27.664138    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:28.081348 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:28.081261    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"]
Nov 04 01:53:28.081532 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:28.081506    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy" containerID="cri-o://c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084" gracePeriod=30
Nov 04 01:53:28.250322 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:28.250292    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:30.330363 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:30.330322    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:30.573687 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:30.573653    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:32.284252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:32.284212    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:33.345460 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.345439    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:53:33.379901 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.379843    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.379901 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.379884    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.379936    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.379988    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380057    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380213 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380101    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380213 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380128    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380213 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380170    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380213 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380097    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.380409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380220    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.380409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380246    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data\") pod \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\" (UID: \"42599b98-ce4c-461d-9cfa-b3f3d38d9153\") "
Nov 04 01:53:33.380409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380394    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.380537 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380432    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:33.380718 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.380686    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.381075 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.381051    5039 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.381158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.381079    5039 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.381158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.381093    5039 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.381158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.381109    5039 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.381158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.381123    5039 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.382728 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.382703    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.382858 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.382840    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "kube-api-access-zb8wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:33.382949 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.382919    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:53:33.383021 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.383001    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token" pod "42599b98-ce4c-461d-9cfa-b3f3d38d9153" (UID: "42599b98-ce4c-461d-9cfa-b3f3d38d9153"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:33.482682 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.482652    5039 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.482682 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.482677    5039 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.482682 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.482686    5039 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.482912 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.482700    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb8wf\" (UniqueName: \"kubernetes.io/projected/42599b98-ce4c-461d-9cfa-b3f3d38d9153-kube-api-access-zb8wf\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.674308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.674217    5039 generic.go:356] "Generic (PLEG): container finished" podID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerID="c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084" exitCode=0
Nov 04 01:53:33.674476 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.674455    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"
Nov 04 01:53:33.674557 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.674516    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" event={"ID":"42599b98-ce4c-461d-9cfa-b3f3d38d9153","Type":"ContainerDied","Data":"c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084"}
Nov 04 01:53:33.674640 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.674622    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g" event={"ID":"42599b98-ce4c-461d-9cfa-b3f3d38d9153","Type":"ContainerDied","Data":"687adbfc70a57d9b5a87b54f94b29edffefdac8e627f69bdc22d98c407ecd30e"}
Nov 04 01:53:33.674686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.674657    5039 scope.go:122] "RemoveContainer" containerID="c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084"
Nov 04 01:53:33.688424 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.688400    5039 scope.go:122] "RemoveContainer" containerID="c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084"
Nov 04 01:53:33.688755 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:53:33.688734    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084\": container with ID starting with c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084 not found: ID does not exist" containerID="c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084"
Nov 04 01:53:33.688826 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.688762    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084"} err="failed to get container status \"c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084\": rpc error: code = NotFound desc = could not find container \"c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084\": container with ID starting with c4e32e6225ff7056ef50f925b4d696f55a93b35a602c56e3157730cc16bbd084 not found: ID does not exist"
Nov 04 01:53:33.840456 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.840425    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"]
Nov 04 01:53:33.844369 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:33.844344    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g"]
Nov 04 01:53:35.371385 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:35.371354    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:35.407637 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:35.407605    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" path="/var/lib/kubelet/pods/42599b98-ce4c-461d-9cfa-b3f3d38d9153/volumes"
Nov 04 01:53:40.050392 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:40.050352    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:42.284155 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:42.284119    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:44.957011 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:44.956978    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:49.880331 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:49.880274    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:52.283955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:52.283922    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:52.752444 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:53:52.752405    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:53:54.645227 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:54.645194    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:59.413269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:59.413229    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:59.472321 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:53:59.472278    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:01.940722 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:01.940679    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:02.946480 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:02.946426    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:03.209174 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:03.209087    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:03.831479 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:03.831437    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:04.089925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:04.089848    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:04.793127 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:04.793096    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:05.045878 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:05.045786    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:06.264142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:06.264106    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"]
Nov 04 01:54:06.508663 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:06.508628    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:09.411019 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:09.410980    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:54:09.553396 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:09.553359    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:09.798679 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:09.798597    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:10.065615 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:10.065514    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:10.311016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:10.310988    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:11.265518 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.265455    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm" podUID="713734bc-821d-4844-9073-d60237efbdf7" containerName="payload-processing" containerID="cri-o://3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1" gracePeriod=25
Nov 04 01:54:11.541599 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.541553    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:54:11.639518 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.639495    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume\") pod \"713734bc-821d-4844-9073-d60237efbdf7\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") "
Nov 04 01:54:11.639712 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.639535    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s\") pod \"713734bc-821d-4844-9073-d60237efbdf7\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") "
Nov 04 01:54:11.639712 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.639639    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials\") pod \"713734bc-821d-4844-9073-d60237efbdf7\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") "
Nov 04 01:54:11.639712 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.639683    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume\") pod \"713734bc-821d-4844-9073-d60237efbdf7\" (UID: \"713734bc-821d-4844-9073-d60237efbdf7\") "
Nov 04 01:54:11.639902 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.639816    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume" pod "713734bc-821d-4844-9073-d60237efbdf7" (UID: "713734bc-821d-4844-9073-d60237efbdf7"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:11.640058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.640022    5039 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-routing-overlay-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:11.640159 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.640111    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume" pod "713734bc-821d-4844-9073-d60237efbdf7" (UID: "713734bc-821d-4844-9073-d60237efbdf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:11.641968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.641947    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials" pod "713734bc-821d-4844-9073-d60237efbdf7" (UID: "713734bc-821d-4844-9073-d60237efbdf7"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:11.642040 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.641970    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s" pod "713734bc-821d-4844-9073-d60237efbdf7" (UID: "713734bc-821d-4844-9073-d60237efbdf7"). InnerVolumeSpecName "kube-api-access-kv49s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:11.740992 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.740960    5039 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-provider-credentials\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:11.740992 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.740987    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/713734bc-821d-4844-9073-d60237efbdf7-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:11.740992 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.740997    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv49s\" (UniqueName: \"kubernetes.io/projected/713734bc-821d-4844-9073-d60237efbdf7-kube-api-access-kv49s\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:11.986702 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.986667    5039 generic.go:356] "Generic (PLEG): container finished" podID="713734bc-821d-4844-9073-d60237efbdf7" containerID="3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1" exitCode=0
Nov 04 01:54:11.986882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.986715    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"
Nov 04 01:54:11.986882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.986748    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm" event={"ID":"713734bc-821d-4844-9073-d60237efbdf7","Type":"ContainerDied","Data":"3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1"}
Nov 04 01:54:11.986882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.986796    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm" event={"ID":"713734bc-821d-4844-9073-d60237efbdf7","Type":"ContainerDied","Data":"bdef0960c2de227ac11faf52078f47b804949da149ea4ec6599e1f23ba5c52ec"}
Nov 04 01:54:11.986882 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.986819    5039 scope.go:122] "RemoveContainer" containerID="3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1"
Nov 04 01:54:11.996608 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.996582    5039 scope.go:122] "RemoveContainer" containerID="3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1"
Nov 04 01:54:11.996872 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:11.996852    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1\": container with ID starting with 3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1 not found: ID does not exist" containerID="3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1"
Nov 04 01:54:11.996918 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:11.996882    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1"} err="failed to get container status \"3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1\": rpc error: code = NotFound desc = could not find container \"3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1\": container with ID starting with 3258923255a1d44e63ffaa5dd4bfcad0dbbc2cb46fef28d2b2e46ecba0b984e1 not found: ID does not exist"
Nov 04 01:54:12.150350 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:12.150314    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"]
Nov 04 01:54:12.199817 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:12.199791    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-c2cde5/payload-processing-external-model-e2e-extmod-b-c2cde5-7ccbwmvbm"]
Nov 04 01:54:13.406216 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:13.406185    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="713734bc-821d-4844-9073-d60237efbdf7" path="/var/lib/kubelet/pods/713734bc-821d-4844-9073-d60237efbdf7/volumes"
Nov 04 01:54:14.336203 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:14.336168    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:15.617123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:15.617097    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:15.865469 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:15.865434    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:16.137904 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:16.137876    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:19.225430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:19.225396    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:19.410478 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:19.410443    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:54:22.162450 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:22.162419    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:22.599979 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:22.599948    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"]
Nov 04 01:54:22.710325 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:22.710292    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"]
Nov 04 01:54:22.752846 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:22.752815    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:54:23.062822 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.062746    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:23.218286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.218245    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"]
Nov 04 01:54:23.218783 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.218539    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" podUID="6ed265b4-85d7-4663-89c5-d7a404871a75" containerName="maas-api" containerID="cri-o://62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97" gracePeriod=30
Nov 04 01:54:23.316225 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.316142    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:23.520442 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.520417    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:54:23.571814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.571701    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:23.665310 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.665264    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd\" (UniqueName: \"kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd\") pod \"6ed265b4-85d7-4663-89c5-d7a404871a75\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") "
Nov 04 01:54:23.665603 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.665562    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\") pod \"6ed265b4-85d7-4663-89c5-d7a404871a75\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") "
Nov 04 01:54:23.665744 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.665729    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca\") pod \"6ed265b4-85d7-4663-89c5-d7a404871a75\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") "
Nov 04 01:54:23.665889 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.665877    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs\") pod \"6ed265b4-85d7-4663-89c5-d7a404871a75\" (UID: \"6ed265b4-85d7-4663-89c5-d7a404871a75\") "
Nov 04 01:54:23.667094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.667061    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca" pod "6ed265b4-85d7-4663-89c5-d7a404871a75" (UID: "6ed265b4-85d7-4663-89c5-d7a404871a75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:23.668783 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.668759    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls" pod "6ed265b4-85d7-4663-89c5-d7a404871a75" (UID: "6ed265b4-85d7-4663-89c5-d7a404871a75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:54:23.668934 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.668838    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd" pod "6ed265b4-85d7-4663-89c5-d7a404871a75" (UID: "6ed265b4-85d7-4663-89c5-d7a404871a75"). InnerVolumeSpecName "kube-api-access-tzsrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:23.669705 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.669166    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs" pod "6ed265b4-85d7-4663-89c5-d7a404871a75" (UID: "6ed265b4-85d7-4663-89c5-d7a404871a75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:54:23.767285 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.767248    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:23.767439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.767289    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzsrd\" (UniqueName: \"kubernetes.io/projected/6ed265b4-85d7-4663-89c5-d7a404871a75-kube-api-access-tzsrd\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:23.767439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.767305    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ed265b4-85d7-4663-89c5-d7a404871a75-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:23.767439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:23.767318    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed265b4-85d7-4663-89c5-d7a404871a75-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:24.065551 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.065517    5039 generic.go:356] "Generic (PLEG): container finished" podID="6ed265b4-85d7-4663-89c5-d7a404871a75" containerID="62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97" exitCode=0
Nov 04 01:54:24.065762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.065556    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"
Nov 04 01:54:24.065762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.065605    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" event={"ID":"6ed265b4-85d7-4663-89c5-d7a404871a75","Type":"ContainerDied","Data":"62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97"}
Nov 04 01:54:24.065762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.065669    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p" event={"ID":"6ed265b4-85d7-4663-89c5-d7a404871a75","Type":"ContainerDied","Data":"1d0c77df0e942148a69555e86974ce3dec67f08ac461d874597606ab7a4c0724"}
Nov 04 01:54:24.065762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.065698    5039 scope.go:122] "RemoveContainer" containerID="62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97"
Nov 04 01:54:24.075885 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.075868    5039 scope.go:122] "RemoveContainer" containerID="62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97"
Nov 04 01:54:24.076135 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:24.076117    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97\": container with ID starting with 62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97 not found: ID does not exist" containerID="62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97"
Nov 04 01:54:24.076201 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.076141    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97"} err="failed to get container status \"62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97\": rpc error: code = NotFound desc = could not find container \"62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97\": container with ID starting with 62b82b868291dc082660b8f6ef0c97b6541b02e5f0a9dd1f793b482644838b97 not found: ID does not exist"
Nov 04 01:54:24.234292 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.234254    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"]
Nov 04 01:54:24.281583 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:24.281536    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c62113-85b98fc58f-jwm8p"]
Nov 04 01:54:25.272543 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:25.272492    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:25.406731 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:25.406701    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ed265b4-85d7-4663-89c5-d7a404871a75" path="/var/lib/kubelet/pods/6ed265b4-85d7-4663-89c5-d7a404871a75/volumes"
Nov 04 01:54:26.356625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:26.356553    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:27.306695 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:27.306665    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:27.601076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:27.601045    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt" podUID="21560ae3-d894-4d5d-9e16-bbbff6c3c215" containerName="payload-processing" containerID="cri-o://fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0" gracePeriod=25
Nov 04 01:54:27.716199 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:27.716155    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz" podUID="15ddc37a-dfea-4e44-b622-d9be7c3dba5a" containerName="payload-pre-processing" containerID="cri-o://0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe" gracePeriod=25
Nov 04 01:54:27.843041 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:27.842843    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:27.866686 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:27.866026    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:54:28.002603 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.002559    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:54:28.007225 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.007204    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9\" (UniqueName: \"kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9\") pod \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") "
Nov 04 01:54:28.007304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.007278    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume\") pod \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\" (UID: \"21560ae3-d894-4d5d-9e16-bbbff6c3c215\") "
Nov 04 01:54:28.007652 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.007630    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume" pod "21560ae3-d894-4d5d-9e16-bbbff6c3c215" (UID: "21560ae3-d894-4d5d-9e16-bbbff6c3c215"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:28.009414 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.009395    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9" pod "21560ae3-d894-4d5d-9e16-bbbff6c3c215" (UID: "21560ae3-d894-4d5d-9e16-bbbff6c3c215"). InnerVolumeSpecName "kube-api-access-xk6k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:28.090774 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.090747    5039 generic.go:356] "Generic (PLEG): container finished" podID="15ddc37a-dfea-4e44-b622-d9be7c3dba5a" containerID="0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe" exitCode=0
Nov 04 01:54:28.090981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.090863    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"
Nov 04 01:54:28.090981 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.090904    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz" event={"ID":"15ddc37a-dfea-4e44-b622-d9be7c3dba5a","Type":"ContainerDied","Data":"0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe"}
Nov 04 01:54:28.091094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.090979    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz" event={"ID":"15ddc37a-dfea-4e44-b622-d9be7c3dba5a","Type":"ContainerDied","Data":"85be47d827cd8cdf11ac27ee55e2d52702c16b09b2623eb71130b722f0731a22"}
Nov 04 01:54:28.091094 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.091010    5039 scope.go:122] "RemoveContainer" containerID="0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe"
Nov 04 01:54:28.092368 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.092341    5039 generic.go:356] "Generic (PLEG): container finished" podID="21560ae3-d894-4d5d-9e16-bbbff6c3c215" containerID="fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0" exitCode=0
Nov 04 01:54:28.092483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.092389    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt" event={"ID":"21560ae3-d894-4d5d-9e16-bbbff6c3c215","Type":"ContainerDied","Data":"fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0"}
Nov 04 01:54:28.092483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.092401    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"
Nov 04 01:54:28.092483 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.092418    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt" event={"ID":"21560ae3-d894-4d5d-9e16-bbbff6c3c215","Type":"ContainerDied","Data":"fa24e9bc1873f81f41d7e304ccf1285379a7403b2d68ed259c0d9871db8a18c3"}
Nov 04 01:54:28.101781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.101766    5039 scope.go:122] "RemoveContainer" containerID="0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe"
Nov 04 01:54:28.102064 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:28.102046    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe\": container with ID starting with 0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe not found: ID does not exist" containerID="0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe"
Nov 04 01:54:28.102154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.102075    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe"} err="failed to get container status \"0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe\": rpc error: code = NotFound desc = could not find container \"0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe\": container with ID starting with 0619f74c0c75853bd6e5f0905ae20d722200262e3399d4d6d02db1f3decc1cbe not found: ID does not exist"
Nov 04 01:54:28.102154 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.102097    5039 scope.go:122] "RemoveContainer" containerID="fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0"
Nov 04 01:54:28.108293 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.108273    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh\" (UniqueName: \"kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh\") pod \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") "
Nov 04 01:54:28.108423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.108411    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume\") pod \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\" (UID: \"15ddc37a-dfea-4e44-b622-d9be7c3dba5a\") "
Nov 04 01:54:28.108676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.108659    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk6k9\" (UniqueName: \"kubernetes.io/projected/21560ae3-d894-4d5d-9e16-bbbff6c3c215-kube-api-access-xk6k9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:28.108764 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.108677    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21560ae3-d894-4d5d-9e16-bbbff6c3c215-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:28.108814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.108767    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume" pod "15ddc37a-dfea-4e44-b622-d9be7c3dba5a" (UID: "15ddc37a-dfea-4e44-b622-d9be7c3dba5a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:28.110391 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.110370    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh" pod "15ddc37a-dfea-4e44-b622-d9be7c3dba5a" (UID: "15ddc37a-dfea-4e44-b622-d9be7c3dba5a"). InnerVolumeSpecName "kube-api-access-krfqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:28.110725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.110710    5039 scope.go:122] "RemoveContainer" containerID="fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0"
Nov 04 01:54:28.110968 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:28.110951    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0\": container with ID starting with fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0 not found: ID does not exist" containerID="fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0"
Nov 04 01:54:28.111009 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.110973    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0"} err="failed to get container status \"fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0\": rpc error: code = NotFound desc = could not find container \"fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0\": container with ID starting with fece59639e0e27111b475f4a067e1ab9f8b63c96e25ddbc540aa9cb5cbc20bd0 not found: ID does not exist"
Nov 04 01:54:28.209926 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.209843    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:28.209926 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.209871    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krfqh\" (UniqueName: \"kubernetes.io/projected/15ddc37a-dfea-4e44-b622-d9be7c3dba5a-kube-api-access-krfqh\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:28.234056 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.234025    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"]
Nov 04 01:54:28.239625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.239596    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt"]
Nov 04 01:54:28.509934 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.509858    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"]
Nov 04 01:54:28.514315 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.514293    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz"]
Nov 04 01:54:28.631329 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:28.631298    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:29.239754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.239720    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:29.407000 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.406967    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15ddc37a-dfea-4e44-b622-d9be7c3dba5a" path="/var/lib/kubelet/pods/15ddc37a-dfea-4e44-b622-d9be7c3dba5a/volumes"
Nov 04 01:54:29.407486 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.407469    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21560ae3-d894-4d5d-9e16-bbbff6c3c215" path="/var/lib/kubelet/pods/21560ae3-d894-4d5d-9e16-bbbff6c3c215/volumes"
Nov 04 01:54:29.410800 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.410775    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:54:29.530632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.530535    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:54:29.533984 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.533942    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:54:29.534286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:29.534265    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:30.355970 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:30.355938    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:32.957393 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:32.957359    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:33.218817 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:33.218741    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:33.899015 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:33.898983    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:34.143843 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:34.143680    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:35.490990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:35.490944    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:35.741400 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:35.741312    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:36.031046 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:36.030959    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:36.814485 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:36.814433    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:37.684235 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:37.684200    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:37.932558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:37.932524    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:38.166588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.166542    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"]
Nov 04 01:54:38.167754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167726    5039 cpu_manager.go:414] "containerMap: removing container" podUID="21560ae3-d894-4d5d-9e16-bbbff6c3c215" containerName="payload-processing"
Nov 04 01:54:38.167754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167753    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21560ae3-d894-4d5d-9e16-bbbff6c3c215" containerName="payload-processing"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167771    5039 cpu_manager.go:414] "containerMap: removing container" podUID="15ddc37a-dfea-4e44-b622-d9be7c3dba5a" containerName="payload-pre-processing"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167779    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15ddc37a-dfea-4e44-b622-d9be7c3dba5a" containerName="payload-pre-processing"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167807    5039 cpu_manager.go:414] "containerMap: removing container" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167815    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167836    5039 cpu_manager.go:414] "containerMap: removing container" podUID="713734bc-821d-4844-9073-d60237efbdf7" containerName="payload-processing"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167843    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="713734bc-821d-4844-9073-d60237efbdf7" containerName="payload-processing"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167878    5039 cpu_manager.go:414] "containerMap: removing container" podUID="6ed265b4-85d7-4663-89c5-d7a404871a75" containerName="maas-api"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167885    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ed265b4-85d7-4663-89c5-d7a404871a75" containerName="maas-api"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167896    5039 cpu_manager.go:414] "containerMap: removing container" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" containerName="payload-processing"
Nov 04 01:54:38.167962 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.167903    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" containerName="payload-processing"
Nov 04 01:54:38.168430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.168019    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="42599b98-ce4c-461d-9cfa-b3f3d38d9153" containerName="istio-proxy"
Nov 04 01:54:38.168430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.168033    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="713734bc-821d-4844-9073-d60237efbdf7" containerName="payload-processing"
Nov 04 01:54:38.168430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.168049    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ed265b4-85d7-4663-89c5-d7a404871a75" containerName="maas-api"
Nov 04 01:54:38.168430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.168057    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="65b4910a-7de8-4924-b37c-e98b8b324615" containerName="payload-processing"
Nov 04 01:54:38.168430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.168068    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="21560ae3-d894-4d5d-9e16-bbbff6c3c215" containerName="payload-processing"
Nov 04 01:54:38.168430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.168076    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="15ddc37a-dfea-4e44-b622-d9be7c3dba5a" containerName="payload-pre-processing"
Nov 04 01:54:38.174532 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.174513    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.211971 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.211917    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lq62\" (UniqueName: \"kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62\") pod \"payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.212103 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.212006    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.215151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.215129    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"]
Nov 04 01:54:38.261742 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.261702    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-27bb49\""
Nov 04 01:54:38.261916 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.261759    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-27bb49-dockercfg-j4nwt\""
Nov 04 01:54:38.313763 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.313734    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lq62\" (UniqueName: \"kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62\") pod \"payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.313942 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.313794    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.314454 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.314434    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.317177 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.317157    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"]
Nov 04 01:54:38.321151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.321134    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.374386 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.374357    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lq62\" (UniqueName: \"kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62\") pod \"payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.380107 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.380080    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"]
Nov 04 01:54:38.415282 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.415257    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpknk\" (UniqueName: \"kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk\") pod \"payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.415393 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.415314    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.489201 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.489140    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:38.516505 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.516474    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpknk\" (UniqueName: \"kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk\") pod \"payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.516635 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.516552    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.517261 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.517238    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.579217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.579181    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpknk\" (UniqueName: \"kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk\") pod \"payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.636201 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.636159    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:38.869805 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:38.869774    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"]
Nov 04 01:54:38.871356 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:54:38.871326    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf71f118_5695_42f2_9523_dbce2faa9f33.slice/crio-332f357bf4b9c3957657c3ef4c2d36a90ecd4449e932cd15797c2497f76480d8 WatchSource:0}: Error finding container 332f357bf4b9c3957657c3ef4c2d36a90ecd4449e932cd15797c2497f76480d8: Status 404 returned error can't find the container with id 332f357bf4b9c3957657c3ef4c2d36a90ecd4449e932cd15797c2497f76480d8
Nov 04 01:54:39.012977 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.012952    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"]
Nov 04 01:54:39.014361 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:54:39.014338    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4d37c6_9c9f_4d07_96e1_8fb4af742d1d.slice/crio-1f628da962e0160f13212b8dd5010a30d0349ada2dae2a8c85f542853baa217a WatchSource:0}: Error finding container 1f628da962e0160f13212b8dd5010a30d0349ada2dae2a8c85f542853baa217a: Status 404 returned error can't find the container with id 1f628da962e0160f13212b8dd5010a30d0349ada2dae2a8c85f542853baa217a
Nov 04 01:54:39.182469 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.182372    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b" event={"ID":"af71f118-5695-42f2-9523-dbce2faa9f33","Type":"ContainerStarted","Data":"0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29"}
Nov 04 01:54:39.182469 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.182438    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:39.182693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.182476    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b" event={"ID":"af71f118-5695-42f2-9523-dbce2faa9f33","Type":"ContainerStarted","Data":"332f357bf4b9c3957657c3ef4c2d36a90ecd4449e932cd15797c2497f76480d8"}
Nov 04 01:54:39.184108 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.184082    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz" event={"ID":"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d","Type":"ContainerStarted","Data":"cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8"}
Nov 04 01:54:39.184224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.184120    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz" event={"ID":"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d","Type":"ContainerStarted","Data":"1f628da962e0160f13212b8dd5010a30d0349ada2dae2a8c85f542853baa217a"}
Nov 04 01:54:39.184224 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.184183    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:39.297976 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.297921    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b" podStartSLOduration=1.297905648 podStartE2EDuration="1.297905648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:39.296662283 +0000 UTC m=+1210.012849140" watchObservedRunningTime="2026-11-04 01:54:39.297905648 +0000 UTC m=+1210.014092533"
Nov 04 01:54:39.410654 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.410616    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:54:39.413101 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:39.413058    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz" podStartSLOduration=1.41304513 podStartE2EDuration="1.41304513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:39.411436372 +0000 UTC m=+1210.127623228" watchObservedRunningTime="2026-11-04 01:54:39.41304513 +0000 UTC m=+1210.129231985"
Nov 04 01:54:40.925848 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:40.925812    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:43.244901 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:43.244867    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:44.464551 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:44.464519    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"]
Nov 04 01:54:44.573505 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:44.573472    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"]
Nov 04 01:54:44.941244 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:44.941212    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:45.013598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.013553    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"]
Nov 04 01:54:45.013797 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.013773    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" podUID="f1cbe519-06cc-42da-abd0-f96acc05dc61" containerName="maas-api" containerID="cri-o://0c69e8809846d9fc016471547560e28dccfce2d7721802198d2b780984e61599" gracePeriod=30
Nov 04 01:54:45.199745 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.198653    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:45.229797 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.229762    5039 generic.go:356] "Generic (PLEG): container finished" podID="f1cbe519-06cc-42da-abd0-f96acc05dc61" containerID="0c69e8809846d9fc016471547560e28dccfce2d7721802198d2b780984e61599" exitCode=0
Nov 04 01:54:45.229997 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.229978    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" event={"ID":"f1cbe519-06cc-42da-abd0-f96acc05dc61","Type":"ContainerDied","Data":"0c69e8809846d9fc016471547560e28dccfce2d7721802198d2b780984e61599"}
Nov 04 01:54:45.286183 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.286159    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:54:45.398448 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.398418    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\") pod \"f1cbe519-06cc-42da-abd0-f96acc05dc61\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") "
Nov 04 01:54:45.398659 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.398486    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs\") pod \"f1cbe519-06cc-42da-abd0-f96acc05dc61\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") "
Nov 04 01:54:45.398659 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.398601    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88\" (UniqueName: \"kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88\") pod \"f1cbe519-06cc-42da-abd0-f96acc05dc61\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") "
Nov 04 01:54:45.398795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.398668    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca\") pod \"f1cbe519-06cc-42da-abd0-f96acc05dc61\" (UID: \"f1cbe519-06cc-42da-abd0-f96acc05dc61\") "
Nov 04 01:54:45.399050 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.399023    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca" pod "f1cbe519-06cc-42da-abd0-f96acc05dc61" (UID: "f1cbe519-06cc-42da-abd0-f96acc05dc61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:45.400767 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.400739    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs" pod "f1cbe519-06cc-42da-abd0-f96acc05dc61" (UID: "f1cbe519-06cc-42da-abd0-f96acc05dc61"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:54:45.400856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.400791    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88" pod "f1cbe519-06cc-42da-abd0-f96acc05dc61" (UID: "f1cbe519-06cc-42da-abd0-f96acc05dc61"). InnerVolumeSpecName "kube-api-access-75z88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:45.400856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.400799    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls" pod "f1cbe519-06cc-42da-abd0-f96acc05dc61" (UID: "f1cbe519-06cc-42da-abd0-f96acc05dc61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:54:45.447055 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.447011    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:45.500068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.499992    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:45.500068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.500015    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75z88\" (UniqueName: \"kubernetes.io/projected/f1cbe519-06cc-42da-abd0-f96acc05dc61-kube-api-access-75z88\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:45.500068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.500030    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1cbe519-06cc-42da-abd0-f96acc05dc61-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:45.500068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:45.500043    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1cbe519-06cc-42da-abd0-f96acc05dc61-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:46.238760 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:46.238726    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"
Nov 04 01:54:46.238994 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:46.238762    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6" event={"ID":"f1cbe519-06cc-42da-abd0-f96acc05dc61","Type":"ContainerDied","Data":"af48457be52bf641a460619f04cfb5ea313170ff5045d3aeba63b0ba1dfaf463"}
Nov 04 01:54:46.238994 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:46.238828    5039 scope.go:122] "RemoveContainer" containerID="0c69e8809846d9fc016471547560e28dccfce2d7721802198d2b780984e61599"
Nov 04 01:54:46.403968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:46.403931    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"]
Nov 04 01:54:46.453563 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:46.453531    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-c2cde5-5f79b66578-hk9d6"]
Nov 04 01:54:47.040597 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:47.040542    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:47.413308 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:47.406687    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1cbe519-06cc-42da-abd0-f96acc05dc61" path="/var/lib/kubelet/pods/f1cbe519-06cc-42da-abd0-f96acc05dc61/volumes"
Nov 04 01:54:48.916632 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:48.916595    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:49.411217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.411181    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:54:49.465095 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.465064    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf" podUID="9b5edd0f-d73d-42f6-be0a-4772d180b196" containerName="payload-processing" containerID="cri-o://e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2" gracePeriod=25
Nov 04 01:54:49.573889 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.573851    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc" podUID="1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" containerName="payload-pre-processing" containerID="cri-o://526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b" gracePeriod=25
Nov 04 01:54:49.734426 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.734396    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:54:49.827486 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.827464    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:54:49.842374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.842351    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6\" (UniqueName: \"kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6\") pod \"9b5edd0f-d73d-42f6-be0a-4772d180b196\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") "
Nov 04 01:54:49.842473 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.842419    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume\") pod \"9b5edd0f-d73d-42f6-be0a-4772d180b196\" (UID: \"9b5edd0f-d73d-42f6-be0a-4772d180b196\") "
Nov 04 01:54:49.842792 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.842758    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume" pod "9b5edd0f-d73d-42f6-be0a-4772d180b196" (UID: "9b5edd0f-d73d-42f6-be0a-4772d180b196"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:49.844521 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.844502    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6" pod "9b5edd0f-d73d-42f6-be0a-4772d180b196" (UID: "9b5edd0f-d73d-42f6-be0a-4772d180b196"). InnerVolumeSpecName "kube-api-access-qkrz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:49.943304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.943235    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume\") pod \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") "
Nov 04 01:54:49.943304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.943282    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6\" (UniqueName: \"kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6\") pod \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\" (UID: \"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe\") "
Nov 04 01:54:49.943749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.943532    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume" pod "1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" (UID: "1b7ac2bd-8e28-4401-82e2-bef16d73fdfe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:54:49.943749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.943643    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b5edd0f-d73d-42f6-be0a-4772d180b196-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:49.943749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.943655    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:49.943749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.943666    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkrz6\" (UniqueName: \"kubernetes.io/projected/9b5edd0f-d73d-42f6-be0a-4772d180b196-kube-api-access-qkrz6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:49.945364 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:49.945344    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6" pod "1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" (UID: "1b7ac2bd-8e28-4401-82e2-bef16d73fdfe"). InnerVolumeSpecName "kube-api-access-jwms6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:54:50.044859 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.044833    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwms6\" (UniqueName: \"kubernetes.io/projected/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe-kube-api-access-jwms6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:54:50.192933 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.192909    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:54:50.193298 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.193282    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:54:50.266508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.266434    5039 generic.go:356] "Generic (PLEG): container finished" podID="9b5edd0f-d73d-42f6-be0a-4772d180b196" containerID="e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2" exitCode=0
Nov 04 01:54:50.266508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.266478    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"
Nov 04 01:54:50.266723 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.266515    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf" event={"ID":"9b5edd0f-d73d-42f6-be0a-4772d180b196","Type":"ContainerDied","Data":"e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2"}
Nov 04 01:54:50.266723 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.266600    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf" event={"ID":"9b5edd0f-d73d-42f6-be0a-4772d180b196","Type":"ContainerDied","Data":"2fa79510172d01ffecc44261e071b316fe15add79b11ca81fce94e1b7312a3e3"}
Nov 04 01:54:50.266723 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.266630    5039 scope.go:122] "RemoveContainer" containerID="e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2"
Nov 04 01:54:50.268282 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.268259    5039 generic.go:356] "Generic (PLEG): container finished" podID="1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" containerID="526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b" exitCode=0
Nov 04 01:54:50.268405 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.268305    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"
Nov 04 01:54:50.268405 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.268344    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc" event={"ID":"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe","Type":"ContainerDied","Data":"526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b"}
Nov 04 01:54:50.268405 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.268397    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc" event={"ID":"1b7ac2bd-8e28-4401-82e2-bef16d73fdfe","Type":"ContainerDied","Data":"19316a0e7033a4d4112504704fc16461ea5459860b82875741763628861e2fb0"}
Nov 04 01:54:50.278374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.278354    5039 scope.go:122] "RemoveContainer" containerID="e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2"
Nov 04 01:54:50.278677 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:50.278659    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2\": container with ID starting with e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2 not found: ID does not exist" containerID="e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2"
Nov 04 01:54:50.278764 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.278688    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2"} err="failed to get container status \"e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2\": rpc error: code = NotFound desc = could not find container \"e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2\": container with ID starting with e1915078a33eb54f7af184a84b4009e288e01cc91ab9307c537d40261b5124c2 not found: ID does not exist"
Nov 04 01:54:50.278764 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.278711    5039 scope.go:122] "RemoveContainer" containerID="526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b"
Nov 04 01:54:50.287047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.287030    5039 scope.go:122] "RemoveContainer" containerID="526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b"
Nov 04 01:54:50.287325 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:50.287307    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b\": container with ID starting with 526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b not found: ID does not exist" containerID="526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b"
Nov 04 01:54:50.287394 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.287334    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b"} err="failed to get container status \"526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b\": rpc error: code = NotFound desc = could not find container \"526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b\": container with ID starting with 526f26bfe72d5b9f0ed53033c147160b43487ada584e372a4e0c9af97ff5ab9b not found: ID does not exist"
Nov 04 01:54:50.534957 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.534892    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"]
Nov 04 01:54:50.538441 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.538416    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc"]
Nov 04 01:54:50.646230 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.646204    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"]
Nov 04 01:54:50.650248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:50.650223    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf"]
Nov 04 01:54:51.406076 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:51.406041    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" path="/var/lib/kubelet/pods/1b7ac2bd-8e28-4401-82e2-bef16d73fdfe/volumes"
Nov 04 01:54:51.406449 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:51.406397    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b5edd0f-d73d-42f6-be0a-4772d180b196" path="/var/lib/kubelet/pods/9b5edd0f-d73d-42f6-be0a-4772d180b196/volumes"
Nov 04 01:54:51.939323 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:51.939288    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:52.753541 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:54:52.753506    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:54:55.228663 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:55.228620    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:55.480139 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:55.480050    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:56.837433 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:56.837393    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:57.088379 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:57.088301    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:57.372286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:57.372242    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:57.889369 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:57.889330    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:59.410655 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:54:59.410619    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:02.130911 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:02.130871    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:02.921868 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:02.921834    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:04.272816 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:04.272786    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:06.856744 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:06.856708    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:09.411150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:09.411112    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:10.252432 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:10.252397    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:11.407227 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407200    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"]
Nov 04 01:55:11.407768 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407754    5039 cpu_manager.go:414] "containerMap: removing container" podUID="9b5edd0f-d73d-42f6-be0a-4772d180b196" containerName="payload-processing"
Nov 04 01:55:11.407815 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407768    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b5edd0f-d73d-42f6-be0a-4772d180b196" containerName="payload-processing"
Nov 04 01:55:11.407815 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407784    5039 cpu_manager.go:414] "containerMap: removing container" podUID="f1cbe519-06cc-42da-abd0-f96acc05dc61" containerName="maas-api"
Nov 04 01:55:11.407815 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407789    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1cbe519-06cc-42da-abd0-f96acc05dc61" containerName="maas-api"
Nov 04 01:55:11.407815 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407811    5039 cpu_manager.go:414] "containerMap: removing container" podUID="1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" containerName="payload-pre-processing"
Nov 04 01:55:11.407815 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407816    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" containerName="payload-pre-processing"
Nov 04 01:55:11.407964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407893    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b5edd0f-d73d-42f6-be0a-4772d180b196" containerName="payload-processing"
Nov 04 01:55:11.407964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407902    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1cbe519-06cc-42da-abd0-f96acc05dc61" containerName="maas-api"
Nov 04 01:55:11.407964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.407911    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b7ac2bd-8e28-4401-82e2-bef16d73fdfe" containerName="payload-pre-processing"
Nov 04 01:55:11.412684 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.412663    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.465104 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.465073    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-27bb49-serving-cert\""
Nov 04 01:55:11.516778 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.516748    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"]
Nov 04 01:55:11.567729 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.567695    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.567886 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.567762    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdwmj\" (UniqueName: \"kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.568012 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.567990    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.568113 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.568099    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.669172 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.669065    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.669172 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.669120    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdwmj\" (UniqueName: \"kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.669407 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.669268    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.669621 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.669597    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.669731 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:11.669709    5039 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-serving-cert: secret "maas-api-e2e-extmod-b-27bb49-serving-cert" not found
Nov 04 01:55:11.669797 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:11.669788    5039 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls podName:012d43e8-c980-4860-b4b2-763ae044e099 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:12.169767204 +0000 UTC m=+1242.885954063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls") pod "maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" (UID: "012d43e8-c980-4860-b4b2-763ae044e099") : secret "maas-api-e2e-extmod-b-27bb49-serving-cert" not found
Nov 04 01:55:11.669919 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.669900    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.671886 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.671863    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.693626 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.693556    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:11.732248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.732211    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdwmj\" (UniqueName: \"kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:11.935120 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:11.935027    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:12.175136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:12.175095    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:12.177518 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:12.177487    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\") pod \"maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:12.180554 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:12.180532    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:12.326171 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:12.326105    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:12.704196 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:12.704160    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"]
Nov 04 01:55:12.705814 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:55:12.705784    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012d43e8_c980_4860_b4b2_763ae044e099.slice/crio-85f123b8c6f877d8a5cca4c62e670b61d9edd91ed35319db9ccda8c22b4a73bd WatchSource:0}: Error finding container 85f123b8c6f877d8a5cca4c62e670b61d9edd91ed35319db9ccda8c22b4a73bd: Status 404 returned error can't find the container with id 85f123b8c6f877d8a5cca4c62e670b61d9edd91ed35319db9ccda8c22b4a73bd
Nov 04 01:55:13.309177 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:13.309128    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:13.417797 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:13.417748    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" event={"ID":"012d43e8-c980-4860-b4b2-763ae044e099","Type":"ContainerStarted","Data":"85f123b8c6f877d8a5cca4c62e670b61d9edd91ed35319db9ccda8c22b4a73bd"}
Nov 04 01:55:13.561141 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:13.561029    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:14.397799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:14.397766    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:14.426070 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:14.426033    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" event={"ID":"012d43e8-c980-4860-b4b2-763ae044e099","Type":"ContainerStarted","Data":"4d60828e7713efb4cff9d5b6ba976ae96aff661b29283c5092ee7b2df785f899"}
Nov 04 01:55:14.426238 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:14.426218    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:14.544725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:14.544674    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" podStartSLOduration=2.6036359559999998 podStartE2EDuration="3.544659194s" totalImagesPullingTime="941.023238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:14.542855227 +0000 UTC m=+1245.259042084" watchObservedRunningTime="2026-11-04 01:55:14.544659194 +0000 UTC m=+1245.260846049"
Nov 04 01:55:15.621227 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:15.621179    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"]
Nov 04 01:55:16.088302 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:16.088213    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:16.341172 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:16.341125    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:16.933752 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:16.933712    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:18.504738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:18.504705    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:19.411288 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:19.411241    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:20.437535 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.437507    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:55:20.622479 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.622443    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6" podUID="d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" containerName="payload-processing" containerID="cri-o://76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f" gracePeriod=25
Nov 04 01:55:20.628898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.628863    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:20.894435 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.894413    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:55:20.962481 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.962455    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx\" (UniqueName: \"kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx\") pod \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") "
Nov 04 01:55:20.962630 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.962619    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume\") pod \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\" (UID: \"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf\") "
Nov 04 01:55:20.963006 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.962983    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume" pod "d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" (UID: "d1fa8b9e-3fd1-44b9-8ede-e876217c93cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:20.964719 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.964694    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx" pod "d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" (UID: "d1fa8b9e-3fd1-44b9-8ede-e876217c93cf"). InnerVolumeSpecName "kube-api-access-rmcrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:20.993463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:20.993411    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:21.063854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.063823    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:21.063854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.063845    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmcrx\" (UniqueName: \"kubernetes.io/projected/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf-kube-api-access-rmcrx\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:21.469385 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.469356    5039 generic.go:356] "Generic (PLEG): container finished" podID="d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" containerID="76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f" exitCode=0
Nov 04 01:55:21.469799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.469467    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"
Nov 04 01:55:21.469799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.469501    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6" event={"ID":"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf","Type":"ContainerDied","Data":"76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f"}
Nov 04 01:55:21.469799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.469548    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6" event={"ID":"d1fa8b9e-3fd1-44b9-8ede-e876217c93cf","Type":"ContainerDied","Data":"5d06c7e7ca63a11061a26260f0acaaf53fb07a64ed31791e22876a23599e1717"}
Nov 04 01:55:21.469799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.469598    5039 scope.go:122] "RemoveContainer" containerID="76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f"
Nov 04 01:55:21.479019 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.479000    5039 scope.go:122] "RemoveContainer" containerID="76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f"
Nov 04 01:55:21.479246 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:21.479230    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f\": container with ID starting with 76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f not found: ID does not exist" containerID="76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f"
Nov 04 01:55:21.479292 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.479254    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f"} err="failed to get container status \"76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f\": rpc error: code = NotFound desc = could not find container \"76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f\": container with ID starting with 76f2fe5023e4b9909344d088187b2773056aaa41aa099cba1e1dd962dc5e5d5f not found: ID does not exist"
Nov 04 01:55:21.633243 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.633211    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"]
Nov 04 01:55:21.638833 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:21.638807    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6"]
Nov 04 01:55:22.211325 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:22.211277    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:22.461538 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:22.461440    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:22.755181 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:22.755082    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:55:23.400329 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:23.400297    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:23.407006 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:23.406978    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" path="/var/lib/kubelet/pods/d1fa8b9e-3fd1-44b9-8ede-e876217c93cf/volumes"
Nov 04 01:55:26.688625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:26.688593    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:26.942344 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:26.942255    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:27.191558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:27.191521    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:27.431561 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:27.431510    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:31.756920 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:31.756877    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:32.742655 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:32.742618    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"]
Nov 04 01:55:32.854585 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:32.854547    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"]
Nov 04 01:55:33.236252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:33.236212    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:33.400353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:33.400312    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:33.488694 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:33.488604    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:33.810651 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:33.810555    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:36.897616 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:36.897550    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:37.743718 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:37.743647    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerName="payload-processing" containerID="cri-o://a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45" gracePeriod=25
Nov 04 01:55:37.855274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:37.855229    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerName="payload-pre-processing" containerID="cri-o://aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5" gracePeriod=25
Nov 04 01:55:39.708233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:39.708205    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:39.961047 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:39.960960    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:40.716674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:40.716645    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:41.105892 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:41.105864    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:41.799423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:41.799387    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:43.400680 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:43.400648    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:43.568609 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:43.568552    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:44.575904 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:44.575867    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:45.722243 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:45.722204    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"]
Nov 04 01:55:45.722676 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:45.722409    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" podUID="a962fc40-e1d7-4fc4-bf4c-700a108f15cd" containerName="maas-api" containerID="cri-o://3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572" gracePeriod=30
Nov 04 01:55:45.925619 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:45.925552    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:46.013964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.013933    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:55:46.026065 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.026042    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca\") pod \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") "
Nov 04 01:55:46.026221 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.026110    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs\") pod \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") "
Nov 04 01:55:46.026221 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.026151    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\") pod \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") "
Nov 04 01:55:46.026221 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.026192    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh\" (UniqueName: \"kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh\") pod \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\" (UID: \"a962fc40-e1d7-4fc4-bf4c-700a108f15cd\") "
Nov 04 01:55:46.026454 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.026430    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca" pod "a962fc40-e1d7-4fc4-bf4c-700a108f15cd" (UID: "a962fc40-e1d7-4fc4-bf4c-700a108f15cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:46.026628 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.026611    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:46.028495 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.028466    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh" pod "a962fc40-e1d7-4fc4-bf4c-700a108f15cd" (UID: "a962fc40-e1d7-4fc4-bf4c-700a108f15cd"). InnerVolumeSpecName "kube-api-access-g2jmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:46.028838 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.028817    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs" pod "a962fc40-e1d7-4fc4-bf4c-700a108f15cd" (UID: "a962fc40-e1d7-4fc4-bf4c-700a108f15cd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:46.028915 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.028848    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls" pod "a962fc40-e1d7-4fc4-bf4c-700a108f15cd" (UID: "a962fc40-e1d7-4fc4-bf4c-700a108f15cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:46.127400 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.127361    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:46.127400 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.127399    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:46.127619 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.127414    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2jmh\" (UniqueName: \"kubernetes.io/projected/a962fc40-e1d7-4fc4-bf4c-700a108f15cd-kube-api-access-g2jmh\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:46.619164 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.619136    5039 generic.go:356] "Generic (PLEG): container finished" podID="a962fc40-e1d7-4fc4-bf4c-700a108f15cd" containerID="3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572" exitCode=0
Nov 04 01:55:46.619366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.619181    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"
Nov 04 01:55:46.619366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.619231    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" event={"ID":"a962fc40-e1d7-4fc4-bf4c-700a108f15cd","Type":"ContainerDied","Data":"3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572"}
Nov 04 01:55:46.619366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.619276    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2" event={"ID":"a962fc40-e1d7-4fc4-bf4c-700a108f15cd","Type":"ContainerDied","Data":"e4cce39c46cdcfddd845881edf46a6e6fdd831daa006f59132dd75969a4e76df"}
Nov 04 01:55:46.619366 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.619299    5039 scope.go:122] "RemoveContainer" containerID="3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572"
Nov 04 01:55:46.630650 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.630625    5039 scope.go:122] "RemoveContainer" containerID="3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572"
Nov 04 01:55:46.630974 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:46.630950    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572\": container with ID starting with 3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572 not found: ID does not exist" containerID="3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572"
Nov 04 01:55:46.631087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.630983    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572"} err="failed to get container status \"3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572\": rpc error: code = NotFound desc = could not find container \"3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572\": container with ID starting with 3b8b1703fdeac741c72bd591300b563fcd5050d286d37e91fc5d603666fc3572 not found: ID does not exist"
Nov 04 01:55:46.789086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.789055    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"]
Nov 04 01:55:46.838181 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.838155    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-06099f-6f9f59ddb7-kqtj2"]
Nov 04 01:55:46.857380 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.857351    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"]
Nov 04 01:55:46.857640 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:46.857614    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" podUID="8a60cfce-b37a-428a-bee3-bbe93eb9fb39" containerName="maas-api" containerID="cri-o://1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3" gracePeriod=30
Nov 04 01:55:47.068534 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.065999    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:47.127643 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.127538    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:55:47.136958 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.136930    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca\") pod \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") "
Nov 04 01:55:47.137102 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.137086    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp\" (UniqueName: \"kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp\") pod \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") "
Nov 04 01:55:47.137176 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.137129    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs\") pod \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") "
Nov 04 01:55:47.137176 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.137156    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\") pod \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\" (UID: \"8a60cfce-b37a-428a-bee3-bbe93eb9fb39\") "
Nov 04 01:55:47.137358 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.137332    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca" pod "8a60cfce-b37a-428a-bee3-bbe93eb9fb39" (UID: "8a60cfce-b37a-428a-bee3-bbe93eb9fb39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:47.137501 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.137486    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:47.140429 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.140404    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls" pod "8a60cfce-b37a-428a-bee3-bbe93eb9fb39" (UID: "8a60cfce-b37a-428a-bee3-bbe93eb9fb39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:47.140624 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.140429    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs" pod "8a60cfce-b37a-428a-bee3-bbe93eb9fb39" (UID: "8a60cfce-b37a-428a-bee3-bbe93eb9fb39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:47.140728 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.140436    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp" pod "8a60cfce-b37a-428a-bee3-bbe93eb9fb39" (UID: "8a60cfce-b37a-428a-bee3-bbe93eb9fb39"). InnerVolumeSpecName "kube-api-access-hlqzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:47.238399 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.238363    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlqzp\" (UniqueName: \"kubernetes.io/projected/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-kube-api-access-hlqzp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:47.238399 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.238401    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:47.238612 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.238415    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a60cfce-b37a-428a-bee3-bbe93eb9fb39-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:47.375842 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.375808    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:47.410595 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.410498    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a962fc40-e1d7-4fc4-bf4c-700a108f15cd" path="/var/lib/kubelet/pods/a962fc40-e1d7-4fc4-bf4c-700a108f15cd/volumes"
Nov 04 01:55:47.625003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.624960    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:47.626708 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.626682    5039 generic.go:356] "Generic (PLEG): container finished" podID="8a60cfce-b37a-428a-bee3-bbe93eb9fb39" containerID="1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3" exitCode=0
Nov 04 01:55:47.626854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.626732    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"
Nov 04 01:55:47.626854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.626767    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" event={"ID":"8a60cfce-b37a-428a-bee3-bbe93eb9fb39","Type":"ContainerDied","Data":"1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3"}
Nov 04 01:55:47.626854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.626819    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9" event={"ID":"8a60cfce-b37a-428a-bee3-bbe93eb9fb39","Type":"ContainerDied","Data":"4356d46883e2c3625e29b8c75c0ede671a09b5315e64db08f0f0a494f5b812fe"}
Nov 04 01:55:47.626854 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.626841    5039 scope.go:122] "RemoveContainer" containerID="1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3"
Nov 04 01:55:47.638360 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.638344    5039 scope.go:122] "RemoveContainer" containerID="1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3"
Nov 04 01:55:47.638672 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:47.638654    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3\": container with ID starting with 1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3 not found: ID does not exist" containerID="1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3"
Nov 04 01:55:47.638759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.638678    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3"} err="failed to get container status \"1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3\": rpc error: code = NotFound desc = could not find container \"1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3\": container with ID starting with 1ae1159144a55bba2da8572c6ed26e992279f74d1cecc947d9058064f871bde3 not found: ID does not exist"
Nov 04 01:55:47.794671 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.791106    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"]
Nov 04 01:55:47.839486 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:47.839453    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-795045-85f448c457-b9hz9"]
Nov 04 01:55:48.836312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:48.836279    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:49.406728 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:49.406698    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a60cfce-b37a-428a-bee3-bbe93eb9fb39" path="/var/lib/kubelet/pods/8a60cfce-b37a-428a-bee3-bbe93eb9fb39/volumes"
Nov 04 01:55:52.751221 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:52.751187    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:55:53.400358 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:53.400321    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:53.568593 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:53.568523    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:53.706505 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:53.706427    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:54.576446 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:54.576415    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:54.652433 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:54.652396    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:55.412880 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:55.412836    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:55.713492 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:55.713415    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:55.967663 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:55.967560    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:56.238030 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:56.237941    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:56.491775 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:56.491697    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:56.751660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:56.751583    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:57.028291 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:57.028189    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:57.564980 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:57.564944    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:57.868754 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:57.868556    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:58.035406 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.034473    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:55:58.165292 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.165120    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:55:58.173925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.173899    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume\") pod \"9bef7c76-970c-463d-9eb8-88c042ddee45\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") "
Nov 04 01:55:58.174053 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.173992    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd\" (UniqueName: \"kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd\") pod \"9bef7c76-970c-463d-9eb8-88c042ddee45\" (UID: \"9bef7c76-970c-463d-9eb8-88c042ddee45\") "
Nov 04 01:55:58.174385 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.174350    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume" pod "9bef7c76-970c-463d-9eb8-88c042ddee45" (UID: "9bef7c76-970c-463d-9eb8-88c042ddee45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:58.176332 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.176312    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd" pod "9bef7c76-970c-463d-9eb8-88c042ddee45" (UID: "9bef7c76-970c-463d-9eb8-88c042ddee45"). InnerVolumeSpecName "kube-api-access-5pqpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:58.274614 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.274579    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume\") pod \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") "
Nov 04 01:55:58.274802 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.274700    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88\" (UniqueName: \"kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88\") pod \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\" (UID: \"97e32fc6-86f0-4652-b427-5a993fb5b6bf\") "
Nov 04 01:55:58.274940 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.274916    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bef7c76-970c-463d-9eb8-88c042ddee45-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:58.274940 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.274932    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pqpd\" (UniqueName: \"kubernetes.io/projected/9bef7c76-970c-463d-9eb8-88c042ddee45-kube-api-access-5pqpd\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:58.275111 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.275015    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume" pod "97e32fc6-86f0-4652-b427-5a993fb5b6bf" (UID: "97e32fc6-86f0-4652-b427-5a993fb5b6bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:58.277081 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.277057    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88" pod "97e32fc6-86f0-4652-b427-5a993fb5b6bf" (UID: "97e32fc6-86f0-4652-b427-5a993fb5b6bf"). InnerVolumeSpecName "kube-api-access-28n88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:58.375977 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.375937    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e32fc6-86f0-4652-b427-5a993fb5b6bf-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:58.375977 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.375975    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28n88\" (UniqueName: \"kubernetes.io/projected/97e32fc6-86f0-4652-b427-5a993fb5b6bf-kube-api-access-28n88\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:58.401286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.401190    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:58.697377 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.697301    5039 generic.go:356] "Generic (PLEG): container finished" podID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerID="a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45" exitCode=0
Nov 04 01:55:58.697377 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.697354    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"
Nov 04 01:55:58.697674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.697383    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" event={"ID":"9bef7c76-970c-463d-9eb8-88c042ddee45","Type":"ContainerDied","Data":"a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45"}
Nov 04 01:55:58.697674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.697442    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz" event={"ID":"9bef7c76-970c-463d-9eb8-88c042ddee45","Type":"ContainerDied","Data":"302e43e7c445b10c03d55ef9b761bce23b1e213491756855e2ff695e937a03b2"}
Nov 04 01:55:58.697674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.697471    5039 scope.go:122] "RemoveContainer" containerID="a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45"
Nov 04 01:55:58.703820 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.703800    5039 generic.go:356] "Generic (PLEG): container finished" podID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerID="aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5" exitCode=0
Nov 04 01:55:58.703939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.703843    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"
Nov 04 01:55:58.703939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.703857    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" event={"ID":"97e32fc6-86f0-4652-b427-5a993fb5b6bf","Type":"ContainerDied","Data":"aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5"}
Nov 04 01:55:58.703939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.703900    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj" event={"ID":"97e32fc6-86f0-4652-b427-5a993fb5b6bf","Type":"ContainerDied","Data":"b9e659a402fd8d03119318a3026b10ff3c94d29ae181f0336e01e9eeaf09dab0"}
Nov 04 01:55:58.708225 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.708205    5039 scope.go:122] "RemoveContainer" containerID="a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45"
Nov 04 01:55:58.708502 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:58.708484    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45\": container with ID starting with a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45 not found: ID does not exist" containerID="a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45"
Nov 04 01:55:58.708621 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.708508    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45"} err="failed to get container status \"a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45\": rpc error: code = NotFound desc = could not find container \"a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45\": container with ID starting with a3dd1259e21fb7035080eebdb38936a506553e6934958ec1473447935626df45 not found: ID does not exist"
Nov 04 01:55:58.708621 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.708525    5039 scope.go:122] "RemoveContainer" containerID="aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5"
Nov 04 01:55:58.717190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.717174    5039 scope.go:122] "RemoveContainer" containerID="aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5"
Nov 04 01:55:58.717424 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:55:58.717401    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5\": container with ID starting with aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5 not found: ID does not exist" containerID="aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5"
Nov 04 01:55:58.717503 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.717428    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5"} err="failed to get container status \"aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5\": rpc error: code = NotFound desc = could not find container \"aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5\": container with ID starting with aaf6fa6839ef522aef996007fb1a6793fd14ad81631ffc16c943e04d19c76ef5 not found: ID does not exist"
Nov 04 01:55:58.861836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.861791    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"]
Nov 04 01:55:58.864679 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.864652    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj"]
Nov 04 01:55:58.953529 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.953449    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:58.975176 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.975144    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"]
Nov 04 01:55:58.980350 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:58.980324    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz"]
Nov 04 01:55:59.207433 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:59.207355    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:59.406469 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:59.406441    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" path="/var/lib/kubelet/pods/97e32fc6-86f0-4652-b427-5a993fb5b6bf/volumes"
Nov 04 01:55:59.406871 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:59.406857    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" path="/var/lib/kubelet/pods/9bef7c76-970c-463d-9eb8-88c042ddee45/volumes"
Nov 04 01:55:59.767357 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:55:59.767313    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:01.824186 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:01.824149    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:02.880629 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.880595    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"]
Nov 04 01:56:02.881317 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881300    5039 cpu_manager.go:414] "containerMap: removing container" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerName="payload-processing"
Nov 04 01:56:02.881359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881320    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerName="payload-processing"
Nov 04 01:56:02.881359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881347    5039 cpu_manager.go:414] "containerMap: removing container" podUID="8a60cfce-b37a-428a-bee3-bbe93eb9fb39" containerName="maas-api"
Nov 04 01:56:02.881359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881356    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a60cfce-b37a-428a-bee3-bbe93eb9fb39" containerName="maas-api"
Nov 04 01:56:02.881447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881367    5039 cpu_manager.go:414] "containerMap: removing container" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerName="payload-pre-processing"
Nov 04 01:56:02.881447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881376    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerName="payload-pre-processing"
Nov 04 01:56:02.881447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881402    5039 cpu_manager.go:414] "containerMap: removing container" podUID="a962fc40-e1d7-4fc4-bf4c-700a108f15cd" containerName="maas-api"
Nov 04 01:56:02.881447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881409    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a962fc40-e1d7-4fc4-bf4c-700a108f15cd" containerName="maas-api"
Nov 04 01:56:02.881447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881440    5039 cpu_manager.go:414] "containerMap: removing container" podUID="d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" containerName="payload-processing"
Nov 04 01:56:02.881613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881447    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" containerName="payload-processing"
Nov 04 01:56:02.881613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881561    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bef7c76-970c-463d-9eb8-88c042ddee45" containerName="payload-processing"
Nov 04 01:56:02.881613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881591    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a60cfce-b37a-428a-bee3-bbe93eb9fb39" containerName="maas-api"
Nov 04 01:56:02.881613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881601    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="97e32fc6-86f0-4652-b427-5a993fb5b6bf" containerName="payload-pre-processing"
Nov 04 01:56:02.881613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881612    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="a962fc40-e1d7-4fc4-bf4c-700a108f15cd" containerName="maas-api"
Nov 04 01:56:02.881758 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.881627    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1fa8b9e-3fd1-44b9-8ede-e876217c93cf" containerName="payload-processing"
Nov 04 01:56:02.886738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.886719    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:02.962089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.962046    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"]
Nov 04 01:56:02.997642 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.997612    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-27bb49\"/\"openshift-service-ca.crt\""
Nov 04 01:56:02.997836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.997682    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-27bb49\"/\"kube-root-ca.crt\""
Nov 04 01:56:02.997836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.997690    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-27bb49\"/\"routing-overlay\""
Nov 04 01:56:02.997836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.997781    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-27bb49\"/\"payload-processing-external-model-plugins-e2e-extmod-b-27bb49\""
Nov 04 01:56:02.997836 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.997785    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-27bb49\"/\"agc-cred-22c2c976ea41\""
Nov 04 01:56:02.998142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:02.998112    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-27bb49\"/\"payload-processing-external-model-e2e--4fac25ca-dockercfg-zqlsd\""
Nov 04 01:56:03.023269 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.023244    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ngbk\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.023381 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.023277    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.023424 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.023392    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.023481 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.023467    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.124988 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.124962    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.125107 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.125013    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ngbk\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.125152 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.125130    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.125258 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.125242    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.125725 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.125698    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.125833 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.125763    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.127634 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.127617    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.185604 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.185520    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ngbk\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk\") pod \"payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") " pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.200343 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.200322    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.400693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.400651    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:03.576248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.576224    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"]
Nov 04 01:56:03.578196 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:56:03.578166    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fb7f92_e194_4b9c_a460_924ecc0d39db.slice/crio-7dfdfcad25dc8106150417ca9634ccd91e46aa280a10ac189dd0e24cf9608b33 WatchSource:0}: Error finding container 7dfdfcad25dc8106150417ca9634ccd91e46aa280a10ac189dd0e24cf9608b33: Status 404 returned error can't find the container with id 7dfdfcad25dc8106150417ca9634ccd91e46aa280a10ac189dd0e24cf9608b33
Nov 04 01:56:03.736751 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.736722    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh" event={"ID":"73fb7f92-e194-4b9c-a460-924ecc0d39db","Type":"ContainerStarted","Data":"d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316"}
Nov 04 01:56:03.736936 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.736773    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh" event={"ID":"73fb7f92-e194-4b9c-a460-924ecc0d39db","Type":"ContainerStarted","Data":"7dfdfcad25dc8106150417ca9634ccd91e46aa280a10ac189dd0e24cf9608b33"}
Nov 04 01:56:03.736936 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.736825    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:03.854751 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:03.854696    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh" podStartSLOduration=1.8546792010000002 podStartE2EDuration="1.854679201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:03.852226148 +0000 UTC m=+1294.568413005" watchObservedRunningTime="2026-11-04 01:56:03.854679201 +0000 UTC m=+1294.570866060"
Nov 04 01:56:04.249430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:04.249345    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:05.863738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:05.863708    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:06.861000 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:06.860960    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"]
Nov 04 01:56:06.968549 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:06.968518    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"]
Nov 04 01:56:07.325916 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:07.325837    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:07.574146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:07.574109    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:09.383669 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:09.383630    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:11.862082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:11.862022    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l" podUID="b87635f6-06e0-44fe-9aed-49dd0f2d5511" containerName="payload-processing" containerID="cri-o://95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a" gracePeriod=25
Nov 04 01:56:11.969605 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:11.969542    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerName="payload-pre-processing" containerID="cri-o://9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06" gracePeriod=25
Nov 04 01:56:12.060596 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.057444    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:12.176510 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.176479    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:56:12.342420 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.341992    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5\" (UniqueName: \"kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5\") pod \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") "
Nov 04 01:56:12.342420 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.342190    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume\") pod \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\" (UID: \"b87635f6-06e0-44fe-9aed-49dd0f2d5511\") "
Nov 04 01:56:12.343584 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.343527    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume" pod "b87635f6-06e0-44fe-9aed-49dd0f2d5511" (UID: "b87635f6-06e0-44fe-9aed-49dd0f2d5511"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:12.400638 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.399731    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5" pod "b87635f6-06e0-44fe-9aed-49dd0f2d5511" (UID: "b87635f6-06e0-44fe-9aed-49dd0f2d5511"). InnerVolumeSpecName "kube-api-access-45kc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:12.444590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.444279    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45kc5\" (UniqueName: \"kubernetes.io/projected/b87635f6-06e0-44fe-9aed-49dd0f2d5511-kube-api-access-45kc5\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:12.444590 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.444316    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87635f6-06e0-44fe-9aed-49dd0f2d5511-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:12.798058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.797979    5039 generic.go:356] "Generic (PLEG): container finished" podID="b87635f6-06e0-44fe-9aed-49dd0f2d5511" containerID="95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a" exitCode=0
Nov 04 01:56:12.798058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.798021    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l" event={"ID":"b87635f6-06e0-44fe-9aed-49dd0f2d5511","Type":"ContainerDied","Data":"95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a"}
Nov 04 01:56:12.798058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.798029    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"
Nov 04 01:56:12.798058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.798059    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l" event={"ID":"b87635f6-06e0-44fe-9aed-49dd0f2d5511","Type":"ContainerDied","Data":"87d99280d8102d3f66e55ff60e0bdad6fcf2eeb913041950a121df07dafa420d"}
Nov 04 01:56:12.798362 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.798080    5039 scope.go:122] "RemoveContainer" containerID="95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a"
Nov 04 01:56:12.809910 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.809885    5039 scope.go:122] "RemoveContainer" containerID="95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a"
Nov 04 01:56:12.810228 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:56:12.810206    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a\": container with ID starting with 95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a not found: ID does not exist" containerID="95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a"
Nov 04 01:56:12.810323 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.810239    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a"} err="failed to get container status \"95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a\": rpc error: code = NotFound desc = could not find container \"95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a\": container with ID starting with 95eed34d20c6590a1dd7c8e5f9c0e78a42a067ae746c1f40e4a814c19478470a not found: ID does not exist"
Nov 04 01:56:12.971439 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.971412    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"]
Nov 04 01:56:12.973583 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:12.973546    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l"]
Nov 04 01:56:13.401087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:13.401055    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:13.406762 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:13.406736    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b87635f6-06e0-44fe-9aed-49dd0f2d5511" path="/var/lib/kubelet/pods/b87635f6-06e0-44fe-9aed-49dd0f2d5511/volumes"
Nov 04 01:56:14.270294 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:14.270260    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:14.745480 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:14.745450    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:56:19.147888 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:19.147842    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:20.197644 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:20.197609    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:22.752103 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:56:22.752067    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:56:23.401001 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:23.400965    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:23.936283 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:23.936250    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:25.400450 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:25.400414    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:27.027239 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:27.027195    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:28.850071 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:28.850038    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:30.198238 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:30.198192    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:32.228807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.228751    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:56:32.329598 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.329553    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x\" (UniqueName: \"kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x\") pod \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") "
Nov 04 01:56:32.329782 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.329659    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume\") pod \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\" (UID: \"b6ebcda0-06c7-4aed-94b1-00389a56b35d\") "
Nov 04 01:56:32.330140 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.330110    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume" pod "b6ebcda0-06c7-4aed-94b1-00389a56b35d" (UID: "b6ebcda0-06c7-4aed-94b1-00389a56b35d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:32.331970 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.331946    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x" pod "b6ebcda0-06c7-4aed-94b1-00389a56b35d" (UID: "b6ebcda0-06c7-4aed-94b1-00389a56b35d"). InnerVolumeSpecName "kube-api-access-n9f7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:32.430636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.430602    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9f7x\" (UniqueName: \"kubernetes.io/projected/b6ebcda0-06c7-4aed-94b1-00389a56b35d-kube-api-access-n9f7x\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:32.430636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.430636    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ebcda0-06c7-4aed-94b1-00389a56b35d-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:32.924972 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.924932    5039 generic.go:356] "Generic (PLEG): container finished" podID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerID="9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06" exitCode=0
Nov 04 01:56:32.925116 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.925091    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" event={"ID":"b6ebcda0-06c7-4aed-94b1-00389a56b35d","Type":"ContainerDied","Data":"9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06"}
Nov 04 01:56:32.925170 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.925141    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8" event={"ID":"b6ebcda0-06c7-4aed-94b1-00389a56b35d","Type":"ContainerDied","Data":"51c1a5891d972c6d2525ce4a260eec5110c92bf1c1683b57b08644c37bda65b2"}
Nov 04 01:56:32.925213 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.925167    5039 scope.go:122] "RemoveContainer" containerID="9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06"
Nov 04 01:56:32.925344 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.925331    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"
Nov 04 01:56:32.936407 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.936135    5039 scope.go:122] "RemoveContainer" containerID="9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06"
Nov 04 01:56:32.936700 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:56:32.936496    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06\": container with ID starting with 9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06 not found: ID does not exist" containerID="9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06"
Nov 04 01:56:32.936700 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.936530    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06"} err="failed to get container status \"9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06\": rpc error: code = NotFound desc = could not find container \"9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06\": container with ID starting with 9ec24b7d26af03886ebd5bc083ee8d75d5ecc180989b098cb137c395f794cc06 not found: ID does not exist"
Nov 04 01:56:32.998591 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:32.998540    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:33.090288 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:33.090250    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"]
Nov 04 01:56:33.138400 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:33.138368    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8"]
Nov 04 01:56:33.406441 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:33.406409    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" path="/var/lib/kubelet/pods/b6ebcda0-06c7-4aed-94b1-00389a56b35d/volumes"
Nov 04 01:56:34.708537 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:34.708503    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:35.400746 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:35.400716    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:35.607362 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:35.607326    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:35.850123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:35.850082    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:37.060063 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:37.060028    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:37.565749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:37.565702    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:38.138946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:38.138917    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:39.915648 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:39.915618    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:42.921072 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:42.921039    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"]
Nov 04 01:56:43.116142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:43.116108    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:44.851638 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:44.851590    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:45.145200 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:45.145109    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:45.400693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:45.400610    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:47.610669 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:47.610632    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:49.632168 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:49.632131    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:51.107993 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:51.107964    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:52.752723 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:56:52.752686    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:56:54.273617 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:54.273562    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:54.530636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:54.530533    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:54.774867 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:54.774841    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:55.400524 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:55.400491    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:56:56.893921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:56.893888    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:57.942615 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:57.942581    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb" podUID="0d94dd6b-1341-4e63-bac9-60739173f300" containerName="main" containerID="cri-o://1023eefe3fd0ad6ffb5fa8b08802c6528a74f7a6f6b822332d31110ad25c41df" gracePeriod=45
Nov 04 01:56:58.073555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.073457    5039 generic.go:356] "Generic (PLEG): container finished" podID="0d94dd6b-1341-4e63-bac9-60739173f300" containerID="1023eefe3fd0ad6ffb5fa8b08802c6528a74f7a6f6b822332d31110ad25c41df" exitCode=0
Nov 04 01:56:58.073555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.073527    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb" event={"ID":"0d94dd6b-1341-4e63-bac9-60739173f300","Type":"ContainerDied","Data":"1023eefe3fd0ad6ffb5fa8b08802c6528a74f7a6f6b822332d31110ad25c41df"}
Nov 04 01:56:58.233694 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.233665    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:56:58.294852 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.294818    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache\") pod \"0d94dd6b-1341-4e63-bac9-60739173f300\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") "
Nov 04 01:56:58.295102 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.295083    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs\") pod \"0d94dd6b-1341-4e63-bac9-60739173f300\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") "
Nov 04 01:56:58.295277 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.295252    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk\" (UniqueName: \"kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk\") pod \"0d94dd6b-1341-4e63-bac9-60739173f300\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") "
Nov 04 01:56:58.295374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.295294    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm\") pod \"0d94dd6b-1341-4e63-bac9-60739173f300\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") "
Nov 04 01:56:58.295374 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.295329    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir\") pod \"0d94dd6b-1341-4e63-bac9-60739173f300\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") "
Nov 04 01:56:58.295476 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.295450    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home\") pod \"0d94dd6b-1341-4e63-bac9-60739173f300\" (UID: \"0d94dd6b-1341-4e63-bac9-60739173f300\") "
Nov 04 01:56:58.295946 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.295924    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home" pod "0d94dd6b-1341-4e63-bac9-60739173f300" (UID: "0d94dd6b-1341-4e63-bac9-60739173f300"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.296105 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.296088    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache" pod "0d94dd6b-1341-4e63-bac9-60739173f300" (UID: "0d94dd6b-1341-4e63-bac9-60739173f300"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.297813 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.297790    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir" pod "0d94dd6b-1341-4e63-bac9-60739173f300" (UID: "0d94dd6b-1341-4e63-bac9-60739173f300"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.298680 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.298651    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs" pod "0d94dd6b-1341-4e63-bac9-60739173f300" (UID: "0d94dd6b-1341-4e63-bac9-60739173f300"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:58.299677 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.299650    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk" pod "0d94dd6b-1341-4e63-bac9-60739173f300" (UID: "0d94dd6b-1341-4e63-bac9-60739173f300"). InnerVolumeSpecName "kube-api-access-79cpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:58.299773 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.299718    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm" pod "0d94dd6b-1341-4e63-bac9-60739173f300" (UID: "0d94dd6b-1341-4e63-bac9-60739173f300"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.397465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.397424    5039 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.397465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.397470    5039 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d94dd6b-1341-4e63-bac9-60739173f300-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.397750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.397489    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79cpk\" (UniqueName: \"kubernetes.io/projected/0d94dd6b-1341-4e63-bac9-60739173f300-kube-api-access-79cpk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.397750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.397504    5039 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.397750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.397519    5039 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.397750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:58.397532    5039 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d94dd6b-1341-4e63-bac9-60739173f300-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.079934 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.079903    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb" event={"ID":"0d94dd6b-1341-4e63-bac9-60739173f300","Type":"ContainerDied","Data":"35a4a14f8b433aadcaa7365a9d3caa3f18664d26b049bdbdfa8e6b4cc356ea03"}
Nov 04 01:56:59.079934 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.079936    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"
Nov 04 01:56:59.080462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.079951    5039 scope.go:122] "RemoveContainer" containerID="1023eefe3fd0ad6ffb5fa8b08802c6528a74f7a6f6b822332d31110ad25c41df"
Nov 04 01:56:59.243277 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.243243    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"]
Nov 04 01:56:59.293027 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.292994    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-w3-fa0149-kserve-5c9bfff67f-cqwzb"]
Nov 04 01:56:59.410245 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.410212    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d94dd6b-1341-4e63-bac9-60739173f300" path="/var/lib/kubelet/pods/0d94dd6b-1341-4e63-bac9-60739173f300/volumes"
Nov 04 01:56:59.586256 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:56:59.586202    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:00.073814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:00.073779    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:00.126346 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:00.126312    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"]
Nov 04 01:57:00.323614 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:00.323561    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:00.579465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:00.579428    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:00.935687 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:00.935645    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:02.945987 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:02.945957    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:03.521526 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.521487    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"]
Nov 04 01:57:03.521755 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.521692    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" podUID="01ba092d-ff3f-4a1e-91d3-6b32095f129c" containerName="maas-api" containerID="cri-o://1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7" gracePeriod=30
Nov 04 01:57:03.726607 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.725666    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:03.825486 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.825217    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:57:03.873068 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.873033    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca\") pod \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") "
Nov 04 01:57:03.873233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.873127    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\") pod \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") "
Nov 04 01:57:03.873233 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.873199    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs\") pod \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") "
Nov 04 01:57:03.873339 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.873233    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm\" (UniqueName: \"kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm\") pod \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\" (UID: \"01ba092d-ff3f-4a1e-91d3-6b32095f129c\") "
Nov 04 01:57:03.873413 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.873388    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca" pod "01ba092d-ff3f-4a1e-91d3-6b32095f129c" (UID: "01ba092d-ff3f-4a1e-91d3-6b32095f129c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:03.873661 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.873635    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ba092d-ff3f-4a1e-91d3-6b32095f129c-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:03.875314 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.875290    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls" pod "01ba092d-ff3f-4a1e-91d3-6b32095f129c" (UID: "01ba092d-ff3f-4a1e-91d3-6b32095f129c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:03.875495 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.875478    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm" pod "01ba092d-ff3f-4a1e-91d3-6b32095f129c" (UID: "01ba092d-ff3f-4a1e-91d3-6b32095f129c"). InnerVolumeSpecName "kube-api-access-69btm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:03.875636 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.875615    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs" pod "01ba092d-ff3f-4a1e-91d3-6b32095f129c" (UID: "01ba092d-ff3f-4a1e-91d3-6b32095f129c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:57:03.974190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.974163    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:03.974190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.974187    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69btm\" (UniqueName: \"kubernetes.io/projected/01ba092d-ff3f-4a1e-91d3-6b32095f129c-kube-api-access-69btm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:03.974651 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:03.974206    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01ba092d-ff3f-4a1e-91d3-6b32095f129c-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:04.109527 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.109495    5039 generic.go:356] "Generic (PLEG): container finished" podID="01ba092d-ff3f-4a1e-91d3-6b32095f129c" containerID="1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7" exitCode=0
Nov 04 01:57:04.109730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.109550    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"
Nov 04 01:57:04.109730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.109626    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" event={"ID":"01ba092d-ff3f-4a1e-91d3-6b32095f129c","Type":"ContainerDied","Data":"1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7"}
Nov 04 01:57:04.109730 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.109696    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh" event={"ID":"01ba092d-ff3f-4a1e-91d3-6b32095f129c","Type":"ContainerDied","Data":"6cc24ec983b4d5203c2ec0ca7112b33085434511b114ad0cdb907569142d9396"}
Nov 04 01:57:04.109856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.109741    5039 scope.go:122] "RemoveContainer" containerID="1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7"
Nov 04 01:57:04.119991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.119978    5039 scope.go:122] "RemoveContainer" containerID="1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7"
Nov 04 01:57:04.120293 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:04.120269    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7\": container with ID starting with 1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7 not found: ID does not exist" containerID="1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7"
Nov 04 01:57:04.120369 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.120301    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7"} err="failed to get container status \"1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7\": rpc error: code = NotFound desc = could not find container \"1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7\": container with ID starting with 1999994ce3fb53ab86884395032cb2c08c2f98fe3191cb4e926685073e5d80f7 not found: ID does not exist"
Nov 04 01:57:04.273190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.273149    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"]
Nov 04 01:57:04.277115 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:04.277091    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-fa0149-94b8c9bd8-wq6hh"]
Nov 04 01:57:05.126855 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.126818    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh" podUID="73fb7f92-e194-4b9c-a460-924ecc0d39db" containerName="payload-processing" containerID="cri-o://d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316" gracePeriod=25
Nov 04 01:57:05.390600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.390530    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:57:05.400540 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.400507    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:05.406160 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.406134    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01ba092d-ff3f-4a1e-91d3-6b32095f129c" path="/var/lib/kubelet/pods/01ba092d-ff3f-4a1e-91d3-6b32095f129c/volumes"
Nov 04 01:57:05.488661 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.488632    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials\") pod \"73fb7f92-e194-4b9c-a460-924ecc0d39db\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") "
Nov 04 01:57:05.488830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.488785    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume\") pod \"73fb7f92-e194-4b9c-a460-924ecc0d39db\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") "
Nov 04 01:57:05.488830 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.488822    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk\") pod \"73fb7f92-e194-4b9c-a460-924ecc0d39db\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") "
Nov 04 01:57:05.488964 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.488879    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume\") pod \"73fb7f92-e194-4b9c-a460-924ecc0d39db\" (UID: \"73fb7f92-e194-4b9c-a460-924ecc0d39db\") "
Nov 04 01:57:05.489202 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.489167    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume" pod "73fb7f92-e194-4b9c-a460-924ecc0d39db" (UID: "73fb7f92-e194-4b9c-a460-924ecc0d39db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:05.489338 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.489239    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume" pod "73fb7f92-e194-4b9c-a460-924ecc0d39db" (UID: "73fb7f92-e194-4b9c-a460-924ecc0d39db"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:05.491150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.491125    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk" pod "73fb7f92-e194-4b9c-a460-924ecc0d39db" (UID: "73fb7f92-e194-4b9c-a460-924ecc0d39db"). InnerVolumeSpecName "kube-api-access-9ngbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:05.491286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.491269    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials" pod "73fb7f92-e194-4b9c-a460-924ecc0d39db" (UID: "73fb7f92-e194-4b9c-a460-924ecc0d39db"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:05.532090 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.532051    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:05.589827 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.589802    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:05.589827 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.589826    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ngbk\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-kube-api-access-9ngbk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:05.590004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.589835    5039 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73fb7f92-e194-4b9c-a460-924ecc0d39db-routing-overlay-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:05.590004 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:05.589843    5039 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73fb7f92-e194-4b9c-a460-924ecc0d39db-provider-credentials\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:06.126666 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.126629    5039 generic.go:356] "Generic (PLEG): container finished" podID="73fb7f92-e194-4b9c-a460-924ecc0d39db" containerID="d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316" exitCode=0
Nov 04 01:57:06.126845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.126682    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"
Nov 04 01:57:06.126845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.126718    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh" event={"ID":"73fb7f92-e194-4b9c-a460-924ecc0d39db","Type":"ContainerDied","Data":"d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316"}
Nov 04 01:57:06.126845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.126773    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh" event={"ID":"73fb7f92-e194-4b9c-a460-924ecc0d39db","Type":"ContainerDied","Data":"7dfdfcad25dc8106150417ca9634ccd91e46aa280a10ac189dd0e24cf9608b33"}
Nov 04 01:57:06.126845 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.126800    5039 scope.go:122] "RemoveContainer" containerID="d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316"
Nov 04 01:57:06.136486 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.136471    5039 scope.go:122] "RemoveContainer" containerID="d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316"
Nov 04 01:57:06.136757 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:06.136741    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316\": container with ID starting with d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316 not found: ID does not exist" containerID="d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316"
Nov 04 01:57:06.136807 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.136763    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316"} err="failed to get container status \"d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316\": rpc error: code = NotFound desc = could not find container \"d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316\": container with ID starting with d0274ea5afca8d98930218efde200ca521bec486a0b43d0da45c21795694e316 not found: ID does not exist"
Nov 04 01:57:06.289968 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.289939    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"]
Nov 04 01:57:06.339335 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:06.339310    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-27bb49/payload-processing-external-model-e2e-extmod-b-27bb49-7847kjmsh"]
Nov 04 01:57:07.024328 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:07.024291    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:07.271952 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:07.271925    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:07.406100 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:07.406070    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73fb7f92-e194-4b9c-a460-924ecc0d39db" path="/var/lib/kubelet/pods/73fb7f92-e194-4b9c-a460-924ecc0d39db/volumes"
Nov 04 01:57:09.114555 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:09.114502    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:09.361393 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:09.361362    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:10.244508 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:10.244478    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:11.225376 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:11.225334    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:11.476775 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:11.476690    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:13.098720 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:13.098688    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:13.345463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:13.345424    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:13.593660 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:13.593633    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:15.063492 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:15.063453    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:15.401180 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:15.401138    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:19.210423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:19.210386    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:19.457343 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:19.457301    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:19.849969 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:19.849935    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:20.757534 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:20.757500    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:21.152273 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:21.152238    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"]
Nov 04 01:57:21.237005 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:21.236969    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:21.276589 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:21.260216    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"]
Nov 04 01:57:21.635003 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:21.634965    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:21.885462 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:21.885385    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:22.751935 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:22.751905    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:57:23.872204 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:23.872165    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:24.341544 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:24.341508    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:24.934781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:24.934747    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:25.400921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:25.400888    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:26.153149 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.153067    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z" podUID="116a331b-3176-4ecc-ae7a-e19f0d594911" containerName="payload-processing" containerID="cri-o://415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378" gracePeriod=25
Nov 04 01:57:26.278069 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.278033    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl" podUID="ac7add59-2961-4fbd-b42a-f8dc223ad08d" containerName="payload-pre-processing" containerID="cri-o://57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb" gracePeriod=25
Nov 04 01:57:26.421113 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.421053    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:57:26.493113 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.493086    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz\" (UniqueName: \"kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz\") pod \"116a331b-3176-4ecc-ae7a-e19f0d594911\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") "
Nov 04 01:57:26.493274 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.493195    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume\") pod \"116a331b-3176-4ecc-ae7a-e19f0d594911\" (UID: \"116a331b-3176-4ecc-ae7a-e19f0d594911\") "
Nov 04 01:57:26.493591 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.493538    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume" pod "116a331b-3176-4ecc-ae7a-e19f0d594911" (UID: "116a331b-3176-4ecc-ae7a-e19f0d594911"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:26.493711 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.493653    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/116a331b-3176-4ecc-ae7a-e19f0d594911-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:26.496248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.496223    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz" pod "116a331b-3176-4ecc-ae7a-e19f0d594911" (UID: "116a331b-3176-4ecc-ae7a-e19f0d594911"). InnerVolumeSpecName "kube-api-access-x2mlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:26.531729 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.531709    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:57:26.594475 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.594447    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2mlz\" (UniqueName: \"kubernetes.io/projected/116a331b-3176-4ecc-ae7a-e19f0d594911-kube-api-access-x2mlz\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:26.694987 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.694907    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume\") pod \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") "
Nov 04 01:57:26.695150 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.695059    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8\" (UniqueName: \"kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8\") pod \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\" (UID: \"ac7add59-2961-4fbd-b42a-f8dc223ad08d\") "
Nov 04 01:57:26.695348 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.695319    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume" pod "ac7add59-2961-4fbd-b42a-f8dc223ad08d" (UID: "ac7add59-2961-4fbd-b42a-f8dc223ad08d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:26.697183 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.697163    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8" pod "ac7add59-2961-4fbd-b42a-f8dc223ad08d" (UID: "ac7add59-2961-4fbd-b42a-f8dc223ad08d"). InnerVolumeSpecName "kube-api-access-ttnq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:26.796761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.796733    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttnq8\" (UniqueName: \"kubernetes.io/projected/ac7add59-2961-4fbd-b42a-f8dc223ad08d-kube-api-access-ttnq8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:26.796761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.796756    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7add59-2961-4fbd-b42a-f8dc223ad08d-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:26.831472 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:26.831441    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:27.243185 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.243147    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:27.258277 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.258250    5039 generic.go:356] "Generic (PLEG): container finished" podID="ac7add59-2961-4fbd-b42a-f8dc223ad08d" containerID="57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb" exitCode=0
Nov 04 01:57:27.258463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.258355    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"
Nov 04 01:57:27.258463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.258364    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl" event={"ID":"ac7add59-2961-4fbd-b42a-f8dc223ad08d","Type":"ContainerDied","Data":"57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb"}
Nov 04 01:57:27.258463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.258422    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl" event={"ID":"ac7add59-2961-4fbd-b42a-f8dc223ad08d","Type":"ContainerDied","Data":"a2a4a94d78fd52c411764bb31de1f9f5b6efb7687505464db8caa13f1e6de367"}
Nov 04 01:57:27.258463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.258452    5039 scope.go:122] "RemoveContainer" containerID="57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb"
Nov 04 01:57:27.259886 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.259862    5039 generic.go:356] "Generic (PLEG): container finished" podID="116a331b-3176-4ecc-ae7a-e19f0d594911" containerID="415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378" exitCode=0
Nov 04 01:57:27.259991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.259944    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"
Nov 04 01:57:27.259991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.259969    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z" event={"ID":"116a331b-3176-4ecc-ae7a-e19f0d594911","Type":"ContainerDied","Data":"415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378"}
Nov 04 01:57:27.260100 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.260009    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z" event={"ID":"116a331b-3176-4ecc-ae7a-e19f0d594911","Type":"ContainerDied","Data":"15e56cc7f9cfeda1001031ecffc5b2e8653059fbaaf274d7814bbcfc5d0b8814"}
Nov 04 01:57:27.270156 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.270062    5039 scope.go:122] "RemoveContainer" containerID="57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb"
Nov 04 01:57:27.270394 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:27.270373    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb\": container with ID starting with 57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb not found: ID does not exist" containerID="57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb"
Nov 04 01:57:27.270484 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.270406    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb"} err="failed to get container status \"57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb\": rpc error: code = NotFound desc = could not find container \"57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb\": container with ID starting with 57360c257a91de0f60e650d8f7ca95d352a6417e11a00d71b4b6105ce2778fbb not found: ID does not exist"
Nov 04 01:57:27.270484 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.270449    5039 scope.go:122] "RemoveContainer" containerID="415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378"
Nov 04 01:57:27.279324 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.279305    5039 scope.go:122] "RemoveContainer" containerID="415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378"
Nov 04 01:57:27.279607 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:27.279556    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378\": container with ID starting with 415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378 not found: ID does not exist" containerID="415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378"
Nov 04 01:57:27.279693 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.279616    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378"} err="failed to get container status \"415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378\": rpc error: code = NotFound desc = could not find container \"415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378\": container with ID starting with 415b8e1e860e3fd7fad411e1002285e4f013d16c11dfa41b82b0030341be4378 not found: ID does not exist"
Nov 04 01:57:27.420759 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.420684    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"]
Nov 04 01:57:27.469698 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.469671    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z"]
Nov 04 01:57:27.486398 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.486368    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:27.578574 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.578543    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"]
Nov 04 01:57:27.581775 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:27.581748    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl"]
Nov 04 01:57:29.405671 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:29.405642    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="116a331b-3176-4ecc-ae7a-e19f0d594911" path="/var/lib/kubelet/pods/116a331b-3176-4ecc-ae7a-e19f0d594911/volumes"
Nov 04 01:57:29.406060 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:29.406017    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac7add59-2961-4fbd-b42a-f8dc223ad08d" path="/var/lib/kubelet/pods/ac7add59-2961-4fbd-b42a-f8dc223ad08d/volumes"
Nov 04 01:57:30.009668 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:30.009589    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:34.848950 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:34.848915    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:35.402600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:35.401923    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:39.058021 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:39.057985    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:42.713060 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:42.713025    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:42.959674 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:42.959642    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:43.382464 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:43.382435    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:43.933998 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:43.933964    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:44.773434 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:44.773405    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:45.400638 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:45.400605    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:45.837264 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:45.837179    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:49.957610 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:49.957549    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"]
Nov 04 01:57:50.020087 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:50.020043    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"]
Nov 04 01:57:50.376199 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:50.376160    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:50.626077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:50.626049    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:51.021554 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:51.021518    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:51.269808 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:51.269768    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:52.751500 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:52.751468    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:57:54.400883 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:54.400847    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:54.958395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:54.958361    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b" podUID="af71f118-5695-42f2-9523-dbce2faa9f33" containerName="payload-processing" containerID="cri-o://0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29" gracePeriod=25
Nov 04 01:57:55.020558 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.020517    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz" podUID="4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" containerName="payload-pre-processing" containerID="cri-o://cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8" gracePeriod=25
Nov 04 01:57:55.275902 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.275878    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:57:55.293704 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.293680    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:57:55.377761 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.377733    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk\" (UniqueName: \"kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk\") pod \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") "
Nov 04 01:57:55.377913 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.377770    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume\") pod \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\" (UID: \"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d\") "
Nov 04 01:57:55.377913 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.377843    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume\") pod \"af71f118-5695-42f2-9523-dbce2faa9f33\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") "
Nov 04 01:57:55.377988 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.377916    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62\" (UniqueName: \"kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62\") pod \"af71f118-5695-42f2-9523-dbce2faa9f33\" (UID: \"af71f118-5695-42f2-9523-dbce2faa9f33\") "
Nov 04 01:57:55.378159 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.378136    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume" pod "4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" (UID: "4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:55.378212 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.378196    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume" pod "af71f118-5695-42f2-9523-dbce2faa9f33" (UID: "af71f118-5695-42f2-9523-dbce2faa9f33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:55.380082 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.380063    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk" pod "4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" (UID: "4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d"). InnerVolumeSpecName "kube-api-access-dpknk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:55.380161 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.380092    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62" pod "af71f118-5695-42f2-9523-dbce2faa9f33" (UID: "af71f118-5695-42f2-9523-dbce2faa9f33"). InnerVolumeSpecName "kube-api-access-2lq62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:55.427086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.427054    5039 generic.go:356] "Generic (PLEG): container finished" podID="4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" containerID="cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8" exitCode=0
Nov 04 01:57:55.427621 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.427100    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz" event={"ID":"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d","Type":"ContainerDied","Data":"cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8"}
Nov 04 01:57:55.427799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.427661    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz" event={"ID":"4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d","Type":"ContainerDied","Data":"1f628da962e0160f13212b8dd5010a30d0349ada2dae2a8c85f542853baa217a"}
Nov 04 01:57:55.427799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.427699    5039 scope.go:122] "RemoveContainer" containerID="cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8"
Nov 04 01:57:55.427799 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.427108    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"
Nov 04 01:57:55.431856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.430462    5039 generic.go:356] "Generic (PLEG): container finished" podID="af71f118-5695-42f2-9523-dbce2faa9f33" containerID="0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29" exitCode=0
Nov 04 01:57:55.431856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.430494    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b" event={"ID":"af71f118-5695-42f2-9523-dbce2faa9f33","Type":"ContainerDied","Data":"0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29"}
Nov 04 01:57:55.431856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.430531    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b" event={"ID":"af71f118-5695-42f2-9523-dbce2faa9f33","Type":"ContainerDied","Data":"332f357bf4b9c3957657c3ef4c2d36a90ecd4449e932cd15797c2497f76480d8"}
Nov 04 01:57:55.431856 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.430622    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"
Nov 04 01:57:55.442284 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.442261    5039 scope.go:122] "RemoveContainer" containerID="cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8"
Nov 04 01:57:55.442648 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:55.442615    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8\": container with ID starting with cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8 not found: ID does not exist" containerID="cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8"
Nov 04 01:57:55.442753 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.442651    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8"} err="failed to get container status \"cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8\": rpc error: code = NotFound desc = could not find container \"cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8\": container with ID starting with cb90d6c1ad5e685acc74be4ba4481f07e69a20e5164531d4d3725e66eb737bb8 not found: ID does not exist"
Nov 04 01:57:55.442753 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.442677    5039 scope.go:122] "RemoveContainer" containerID="0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29"
Nov 04 01:57:55.452049 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.452030    5039 scope.go:122] "RemoveContainer" containerID="0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29"
Nov 04 01:57:55.452329 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:57:55.452309    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29\": container with ID starting with 0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29 not found: ID does not exist" containerID="0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29"
Nov 04 01:57:55.452392 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.452336    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29"} err="failed to get container status \"0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29\": rpc error: code = NotFound desc = could not find container \"0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29\": container with ID starting with 0e9a62c6ae33cf11e3446bf989270e085bddd8ad7505b55fae4615ab31695b29 not found: ID does not exist"
Nov 04 01:57:55.478876 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.478855    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpknk\" (UniqueName: \"kubernetes.io/projected/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-kube-api-access-dpknk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:55.478876 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.478876    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:55.478994 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.478886    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af71f118-5695-42f2-9523-dbce2faa9f33-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:55.478994 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.478894    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lq62\" (UniqueName: \"kubernetes.io/projected/af71f118-5695-42f2-9523-dbce2faa9f33-kube-api-access-2lq62\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:55.616890 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.616861    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"]
Nov 04 01:57:55.620665 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.620641    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz"]
Nov 04 01:57:55.728620 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.728593    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"]
Nov 04 01:57:55.733123 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:55.733098    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b"]
Nov 04 01:57:56.259084 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:56.259008    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:56.817749 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:56.817717    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:57.406320 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:57.406292    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" path="/var/lib/kubelet/pods/4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d/volumes"
Nov 04 01:57:57.406671 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:57:57.406659    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af71f118-5695-42f2-9523-dbce2faa9f33" path="/var/lib/kubelet/pods/af71f118-5695-42f2-9523-dbce2faa9f33/volumes"
Nov 04 01:58:00.239018 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.238990    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"]
Nov 04 01:58:00.239428 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.239207    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" podUID="012d43e8-c980-4860-b4b2-763ae044e099" containerName="maas-api" containerID="cri-o://4d60828e7713efb4cff9d5b6ba976ae96aff661b29283c5092ee7b2df785f899" gracePeriod=30
Nov 04 01:58:00.447613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.445516    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:00.469982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.469754    5039 generic.go:356] "Generic (PLEG): container finished" podID="012d43e8-c980-4860-b4b2-763ae044e099" containerID="4d60828e7713efb4cff9d5b6ba976ae96aff661b29283c5092ee7b2df785f899" exitCode=0
Nov 04 01:58:00.469982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.469879    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" event={"ID":"012d43e8-c980-4860-b4b2-763ae044e099","Type":"ContainerDied","Data":"4d60828e7713efb4cff9d5b6ba976ae96aff661b29283c5092ee7b2df785f899"}
Nov 04 01:58:00.536689 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.536665    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:58:00.628894 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.628863    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca\") pod \"012d43e8-c980-4860-b4b2-763ae044e099\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") "
Nov 04 01:58:00.629077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.628951    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj\" (UniqueName: \"kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj\") pod \"012d43e8-c980-4860-b4b2-763ae044e099\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") "
Nov 04 01:58:00.629077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.628989    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\") pod \"012d43e8-c980-4860-b4b2-763ae044e099\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") "
Nov 04 01:58:00.629077 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.629037    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs\") pod \"012d43e8-c980-4860-b4b2-763ae044e099\" (UID: \"012d43e8-c980-4860-b4b2-763ae044e099\") "
Nov 04 01:58:00.629243 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.629210    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca" pod "012d43e8-c980-4860-b4b2-763ae044e099" (UID: "012d43e8-c980-4860-b4b2-763ae044e099"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:00.629472 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.629447    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/012d43e8-c980-4860-b4b2-763ae044e099-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:00.632195 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.632164    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls" pod "012d43e8-c980-4860-b4b2-763ae044e099" (UID: "012d43e8-c980-4860-b4b2-763ae044e099"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:00.632305 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.632205    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj" pod "012d43e8-c980-4860-b4b2-763ae044e099" (UID: "012d43e8-c980-4860-b4b2-763ae044e099"). InnerVolumeSpecName "kube-api-access-cdwmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:00.632305 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.632279    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs" pod "012d43e8-c980-4860-b4b2-763ae044e099" (UID: "012d43e8-c980-4860-b4b2-763ae044e099"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:00.730359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.730331    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdwmj\" (UniqueName: \"kubernetes.io/projected/012d43e8-c980-4860-b4b2-763ae044e099-kube-api-access-cdwmj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:00.730359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.730357    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:00.730359 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:00.730366    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/012d43e8-c980-4860-b4b2-763ae044e099-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:01.476485 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:01.476459    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"
Nov 04 01:58:01.476904 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:01.476489    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" event={"ID":"012d43e8-c980-4860-b4b2-763ae044e099","Type":"ContainerDied","Data":"85f123b8c6f877d8a5cca4c62e670b61d9edd91ed35319db9ccda8c22b4a73bd"}
Nov 04 01:58:01.476904 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:01.476547    5039 scope.go:122] "RemoveContainer" containerID="4d60828e7713efb4cff9d5b6ba976ae96aff661b29283c5092ee7b2df785f899"
Nov 04 01:58:01.639247 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:01.639220    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"]
Nov 04 01:58:01.688592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:01.688513    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56"]
Nov 04 01:58:02.163312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:02.163284    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:02.617539 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:02.617495    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:02.861328 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:02.861290    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:03.406028 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:03.405998    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="012d43e8-c980-4860-b4b2-763ae044e099" path="/var/lib/kubelet/pods/012d43e8-c980-4860-b4b2-763ae044e099/volumes"
Nov 04 01:58:03.432081 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:03.432041    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-27bb49-5989b96bb-tgt56" podUID="012d43e8-c980-4860-b4b2-763ae044e099" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.88:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:58:04.354254 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:04.354224    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:04.400955 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:04.400929    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:04.874239 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:04.874198    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:05.366616 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:05.366521    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:07.214822 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:07.214791    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:10.211421 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:10.211389    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"]
Nov 04 01:58:10.474777 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:10.474644    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:11.415905 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:11.415874    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:12.055109 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:12.055069    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:14.401886 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:14.401844    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:16.840069 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:16.840039    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:17.090948 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:17.090854    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:17.347217 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:17.347138    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:17.923357 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:17.923318    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:21.939781 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:21.939748    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:22.751671 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:58:22.751637    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:58:24.131984 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:24.131942    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:24.401553 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:24.401472    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:25.233259 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.233211    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh" podUID="2300fa57-0a7a-46a5-8e9e-52eb879d6c67" containerName="main" containerID="cri-o://cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2" gracePeriod=45
Nov 04 01:58:25.485865 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.485809    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:58:25.585987 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.585956    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir\") pod \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") "
Nov 04 01:58:25.586166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586025    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9\" (UniqueName: \"kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9\") pod \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") "
Nov 04 01:58:25.586166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586044    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs\" (UniqueName: \"kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs\") pod \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") "
Nov 04 01:58:25.586166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586072    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache\") pod \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") "
Nov 04 01:58:25.586166 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586092    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm\") pod \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") "
Nov 04 01:58:25.586319 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586235    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir" pod "2300fa57-0a7a-46a5-8e9e-52eb879d6c67" (UID: "2300fa57-0a7a-46a5-8e9e-52eb879d6c67"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.586319 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586251    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home\") pod \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\" (UID: \"2300fa57-0a7a-46a5-8e9e-52eb879d6c67\") "
Nov 04 01:58:25.586472 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586419    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home" pod "2300fa57-0a7a-46a5-8e9e-52eb879d6c67" (UID: "2300fa57-0a7a-46a5-8e9e-52eb879d6c67"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.586654 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586624    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache" pod "2300fa57-0a7a-46a5-8e9e-52eb879d6c67" (UID: "2300fa57-0a7a-46a5-8e9e-52eb879d6c67"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.586748 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586658    5039 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.586748 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.586681    5039 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.588480 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.588458    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm" pod "2300fa57-0a7a-46a5-8e9e-52eb879d6c67" (UID: "2300fa57-0a7a-46a5-8e9e-52eb879d6c67"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.588480 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.588467    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs" pod "2300fa57-0a7a-46a5-8e9e-52eb879d6c67" (UID: "2300fa57-0a7a-46a5-8e9e-52eb879d6c67"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:25.588925 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.588910    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9" pod "2300fa57-0a7a-46a5-8e9e-52eb879d6c67" (UID: "2300fa57-0a7a-46a5-8e9e-52eb879d6c67"). InnerVolumeSpecName "kube-api-access-nbvx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:25.614848 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.614823    5039 generic.go:356] "Generic (PLEG): container finished" podID="2300fa57-0a7a-46a5-8e9e-52eb879d6c67" containerID="cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2" exitCode=0
Nov 04 01:58:25.614990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.614881    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh" event={"ID":"2300fa57-0a7a-46a5-8e9e-52eb879d6c67","Type":"ContainerDied","Data":"cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2"}
Nov 04 01:58:25.614990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.614887    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"
Nov 04 01:58:25.614990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.614915    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh" event={"ID":"2300fa57-0a7a-46a5-8e9e-52eb879d6c67","Type":"ContainerDied","Data":"1612dd4ab9869ca778b581734b882993b54c923fc32dd6ead96069ff4f098720"}
Nov 04 01:58:25.614990 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.614930    5039 scope.go:122] "RemoveContainer" containerID="cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2"
Nov 04 01:58:25.628750 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.628729    5039 scope.go:122] "RemoveContainer" containerID="cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2"
Nov 04 01:58:25.629065 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:58:25.629040    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2\": container with ID starting with cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2 not found: ID does not exist" containerID="cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2"
Nov 04 01:58:25.629155 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.629077    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2"} err="failed to get container status \"cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2\": rpc error: code = NotFound desc = could not find container \"cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2\": container with ID starting with cc4898b0b60472d998277351d8637722fe6ad4fb00f5cbfeb8414ba3a41fe0a2 not found: ID does not exist"
Nov 04 01:58:25.687559 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.687518    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbvx9\" (UniqueName: \"kubernetes.io/projected/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-kube-api-access-nbvx9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.687559 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.687560    5039 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.687784 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.687594    5039 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.687784 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.687607    5039 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2300fa57-0a7a-46a5-8e9e-52eb879d6c67-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.782423 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.782277    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"]
Nov 04 01:58:25.785909 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:25.785882    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-w2-82706f-kserve-6598fd7fbb-l5ggh"]
Nov 04 01:58:26.736532 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:26.736503    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:27.408823 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:27.408780    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2300fa57-0a7a-46a5-8e9e-52eb879d6c67" path="/var/lib/kubelet/pods/2300fa57-0a7a-46a5-8e9e-52eb879d6c67/volumes"
Nov 04 01:58:27.735296 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:27.735206    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:30.364062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:30.364030    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:30.730112 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:30.730039    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"]
Nov 04 01:58:30.730264 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:30.730232    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" podUID="d2ae9460-8050-48a0-9632-7187c4a98fc5" containerName="maas-api" containerID="cri-o://c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f" gracePeriod=30
Nov 04 01:58:30.932001 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:30.931965    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:31.003324 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.003294    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:58:31.036716 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.036687    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs\") pod \"d2ae9460-8050-48a0-9632-7187c4a98fc5\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") "
Nov 04 01:58:31.036855 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.036728    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca\") pod \"d2ae9460-8050-48a0-9632-7187c4a98fc5\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") "
Nov 04 01:58:31.036855 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.036828    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9\" (UniqueName: \"kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9\") pod \"d2ae9460-8050-48a0-9632-7187c4a98fc5\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") "
Nov 04 01:58:31.036969 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.036861    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\") pod \"d2ae9460-8050-48a0-9632-7187c4a98fc5\" (UID: \"d2ae9460-8050-48a0-9632-7187c4a98fc5\") "
Nov 04 01:58:31.037110 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.037084    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca" pod "d2ae9460-8050-48a0-9632-7187c4a98fc5" (UID: "d2ae9460-8050-48a0-9632-7187c4a98fc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:31.038920 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.038889    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls" pod "d2ae9460-8050-48a0-9632-7187c4a98fc5" (UID: "d2ae9460-8050-48a0-9632-7187c4a98fc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:31.039016 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.038996    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9" pod "d2ae9460-8050-48a0-9632-7187c4a98fc5" (UID: "d2ae9460-8050-48a0-9632-7187c4a98fc5"). InnerVolumeSpecName "kube-api-access-9f4g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:31.039062 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.039024    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs" pod "d2ae9460-8050-48a0-9632-7187c4a98fc5" (UID: "d2ae9460-8050-48a0-9632-7187c4a98fc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:31.138430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.138390    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f4g9\" (UniqueName: \"kubernetes.io/projected/d2ae9460-8050-48a0-9632-7187c4a98fc5-kube-api-access-9f4g9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:31.138430 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.138432    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:31.138712 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.138446    5039 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2ae9460-8050-48a0-9632-7187c4a98fc5-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:31.138712 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.138458    5039 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2ae9460-8050-48a0-9632-7187c4a98fc5-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:31.650499 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.650460    5039 generic.go:356] "Generic (PLEG): container finished" podID="d2ae9460-8050-48a0-9632-7187c4a98fc5" containerID="c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f" exitCode=0
Nov 04 01:58:31.650939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.650503    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"
Nov 04 01:58:31.650939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.650542    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" event={"ID":"d2ae9460-8050-48a0-9632-7187c4a98fc5","Type":"ContainerDied","Data":"c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f"}
Nov 04 01:58:31.650939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.650621    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq" event={"ID":"d2ae9460-8050-48a0-9632-7187c4a98fc5","Type":"ContainerDied","Data":"3cfcd092902cf9f6e51c9d6729b16b1deb62cbe7d3689c616aaf402355356fde"}
Nov 04 01:58:31.650939 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.650650    5039 scope.go:122] "RemoveContainer" containerID="c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f"
Nov 04 01:58:31.661783 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.661760    5039 scope.go:122] "RemoveContainer" containerID="c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f"
Nov 04 01:58:31.662138 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:58:31.662104    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f\": container with ID starting with c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f not found: ID does not exist" containerID="c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f"
Nov 04 01:58:31.662240 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.662150    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f"} err="failed to get container status \"c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f\": rpc error: code = NotFound desc = could not find container \"c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f\": container with ID starting with c64bff2a7350c7b58d5116d5a1a0dea4fb62376d04d7c5585ff2cb170684ec5f not found: ID does not exist"
Nov 04 01:58:31.814102 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.814022    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"]
Nov 04 01:58:31.820153 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:31.820128    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-82706f-fccd5669-s2vkq"]
Nov 04 01:58:32.647447 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:32.647403    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:33.405325 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:33.405291    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2ae9460-8050-48a0-9632-7187c4a98fc5" path="/var/lib/kubelet/pods/d2ae9460-8050-48a0-9632-7187c4a98fc5/volumes"
Nov 04 01:58:34.401579 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:34.401535    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:35.995058 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:35.995014    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:36.239344 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:36.239303    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:37.358890 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:37.358861    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:42.066297 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:42.066261    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:43.707862 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.707827    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"]
Nov 04 01:58:43.708341 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708325    5039 cpu_manager.go:414] "containerMap: removing container" podUID="4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" containerName="payload-pre-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708343    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" containerName="payload-pre-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708353    5039 cpu_manager.go:414] "containerMap: removing container" podUID="012d43e8-c980-4860-b4b2-763ae044e099" containerName="maas-api"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708358    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="012d43e8-c980-4860-b4b2-763ae044e099" containerName="maas-api"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708364    5039 cpu_manager.go:414] "containerMap: removing container" podUID="b87635f6-06e0-44fe-9aed-49dd0f2d5511" containerName="payload-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708369    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b87635f6-06e0-44fe-9aed-49dd0f2d5511" containerName="payload-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708374    5039 cpu_manager.go:414] "containerMap: removing container" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerName="payload-pre-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708379    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerName="payload-pre-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708387    5039 cpu_manager.go:414] "containerMap: removing container" podUID="ac7add59-2961-4fbd-b42a-f8dc223ad08d" containerName="payload-pre-processing"
Nov 04 01:58:43.708395 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708391    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac7add59-2961-4fbd-b42a-f8dc223ad08d" containerName="payload-pre-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708403    5039 cpu_manager.go:414] "containerMap: removing container" podUID="2300fa57-0a7a-46a5-8e9e-52eb879d6c67" containerName="main"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708408    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2300fa57-0a7a-46a5-8e9e-52eb879d6c67" containerName="main"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708419    5039 cpu_manager.go:414] "containerMap: removing container" podUID="d2ae9460-8050-48a0-9632-7187c4a98fc5" containerName="maas-api"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708423    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2ae9460-8050-48a0-9632-7187c4a98fc5" containerName="maas-api"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708430    5039 cpu_manager.go:414] "containerMap: removing container" podUID="116a331b-3176-4ecc-ae7a-e19f0d594911" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708436    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="116a331b-3176-4ecc-ae7a-e19f0d594911" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708441    5039 cpu_manager.go:414] "containerMap: removing container" podUID="af71f118-5695-42f2-9523-dbce2faa9f33" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708446    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af71f118-5695-42f2-9523-dbce2faa9f33" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708457    5039 cpu_manager.go:414] "containerMap: removing container" podUID="73fb7f92-e194-4b9c-a460-924ecc0d39db" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708461    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73fb7f92-e194-4b9c-a460-924ecc0d39db" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708466    5039 cpu_manager.go:414] "containerMap: removing container" podUID="0d94dd6b-1341-4e63-bac9-60739173f300" containerName="main"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708470    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d94dd6b-1341-4e63-bac9-60739173f300" containerName="main"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708481    5039 cpu_manager.go:414] "containerMap: removing container" podUID="01ba092d-ff3f-4a1e-91d3-6b32095f129c" containerName="maas-api"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708485    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01ba092d-ff3f-4a1e-91d3-6b32095f129c" containerName="maas-api"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708549    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b4d37c6-9c9f-4d07-96e1-8fb4af742d1d" containerName="payload-pre-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708556    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="012d43e8-c980-4860-b4b2-763ae044e099" containerName="maas-api"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708562    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="b87635f6-06e0-44fe-9aed-49dd0f2d5511" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708585    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6ebcda0-06c7-4aed-94b1-00389a56b35d" containerName="payload-pre-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708593    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac7add59-2961-4fbd-b42a-f8dc223ad08d" containerName="payload-pre-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708600    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="2300fa57-0a7a-46a5-8e9e-52eb879d6c67" containerName="main"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708606    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2ae9460-8050-48a0-9632-7187c4a98fc5" containerName="maas-api"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708612    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="116a331b-3176-4ecc-ae7a-e19f0d594911" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708617    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="af71f118-5695-42f2-9523-dbce2faa9f33" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708622    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="73fb7f92-e194-4b9c-a460-924ecc0d39db" containerName="payload-processing"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708627    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d94dd6b-1341-4e63-bac9-60739173f300" containerName="main"
Nov 04 01:58:43.708714 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.708633    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="01ba092d-ff3f-4a1e-91d3-6b32095f129c" containerName="maas-api"
Nov 04 01:58:43.713310 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.713281    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.747898 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.747861    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748010 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.747909    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwnch\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748051 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748023    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748085 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748055    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748087    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748115    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748220 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748203    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748254 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748242    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.748312 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.748298    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.829463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.829274    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"]
Nov 04 01:58:43.849091 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849055    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849099    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849135    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849162    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849248 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849221    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849456 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849248    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwnch\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849456 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849297    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849456 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849334    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849456 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849380    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849536    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849721 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849707    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.849985 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.849963    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.850098 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.850020    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.850241 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.850221    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.851583 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.851551    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.851853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.851835    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.909839 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.909808    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:43.910028 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:43.910004    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwnch\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:44.027929 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.027844    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:44.401625 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.401592    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:44.455842 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.455814    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"]
Nov 04 01:58:44.456907 ip-10-0-140-18 kubenswrapper[5039]: W1104 01:58:44.456871    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20cca625_41e3_4b6b_937f_2a9044e5ff81.slice/crio-4b0ccff1b9b361d7356dc2f3fac24b12495c40275b397483d0338e2ddef9bedd WatchSource:0}: Error finding container 4b0ccff1b9b361d7356dc2f3fac24b12495c40275b397483d0338e2ddef9bedd: Status 404 returned error can't find the container with id 4b0ccff1b9b361d7356dc2f3fac24b12495c40275b397483d0338e2ddef9bedd
Nov 04 01:58:44.459089 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.459052    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:44.459167 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.459131    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:44.459205 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.459176    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:44.723205 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.723112    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" event={"ID":"20cca625-41e3-4b6b-937f-2a9044e5ff81","Type":"ContainerStarted","Data":"7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5"}
Nov 04 01:58:44.723205 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.723157    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" event={"ID":"20cca625-41e3-4b6b-937f-2a9044e5ff81","Type":"ContainerStarted","Data":"4b0ccff1b9b361d7356dc2f3fac24b12495c40275b397483d0338e2ddef9bedd"}
Nov 04 01:58:44.842723 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:44.842669    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podStartSLOduration=1.842655276 podStartE2EDuration="1.842655276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:58:44.84206528 +0000 UTC m=+1455.558252160" watchObservedRunningTime="2026-11-04 01:58:44.842655276 +0000 UTC m=+1455.558842131"
Nov 04 01:58:45.029095 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:45.029006    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:45.030903 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:45.030873    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:45.031116 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:45.030935    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:58:46.028995 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:46.028955    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:46.029637 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:46.029605    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:58:46.970776 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:46.970735    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:47.032927 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:47.032891    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:47.033485 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:47.033457    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:58:48.025751 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:48.025694    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:48.031005 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:48.030977    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:48.031319 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:48.031280    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:58:48.273738 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:48.273711    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:49.030958 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:49.030921    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:49.031252 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:49.031222    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:58:50.037592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:50.037381    5039 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:50.037592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:50.037439    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Nov 04 01:58:51.032923 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.032888    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:51.033215 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.033197    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:51.034028 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.034010    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:51.137791 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.137764    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:51.504449 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.504414    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"]
Nov 04 01:58:51.505304 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.504902    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy" containerID="cri-o://4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15" gracePeriod=30
Nov 04 01:58:51.512393 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.512369    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"]
Nov 04 01:58:51.525657 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.525630    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"]
Nov 04 01:58:51.525887 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.525864    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy" containerID="cri-o://3f26adaa8045ece0db8d30f637144146c2330ae042c6390508e221474aa2f109" gracePeriod=30
Nov 04 01:58:51.644532 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.644493    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:51.789102 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.789019    5039 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" secret="" err="secret \"e2e-worker-w2-82706f-openshift-default-dockercfg-v862d\" not found"
Nov 04 01:58:51.896131 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:51.896097    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:52.150306 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:52.150277    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:52.751389 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:58:52.751350    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:58:52.794357 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:52.794321    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy" containerID="cri-o://7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5" gracePeriod=30
Nov 04 01:58:54.137814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:54.137772    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:54.388013 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:54.387927    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:54.400952 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:54.400924    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:56.782287 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.782228    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:58:56.816379 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.816349    5039 generic.go:356] "Generic (PLEG): container finished" podID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerID="4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15" exitCode=0
Nov 04 01:58:56.816491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.816395    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"
Nov 04 01:58:56.816491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.816395    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" event={"ID":"52880f5a-85cd-403f-95e1-c1f08a3bb9bc","Type":"ContainerDied","Data":"4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15"}
Nov 04 01:58:56.816491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.816455    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq" event={"ID":"52880f5a-85cd-403f-95e1-c1f08a3bb9bc","Type":"ContainerDied","Data":"d6026890224d35d80217e2381c4bd429941f3f647bf9f5b36a5bfaaa65cda95a"}
Nov 04 01:58:56.816491 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.816472    5039 scope.go:122] "RemoveContainer" containerID="4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15"
Nov 04 01:58:56.817853 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.817835    5039 generic.go:356] "Generic (PLEG): container finished" podID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerID="3f26adaa8045ece0db8d30f637144146c2330ae042c6390508e221474aa2f109" exitCode=0
Nov 04 01:58:56.817942 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.817885    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" event={"ID":"bd87e327-135e-4a72-9925-4d28d4a9360a","Type":"ContainerDied","Data":"3f26adaa8045ece0db8d30f637144146c2330ae042c6390508e221474aa2f109"}
Nov 04 01:58:56.817942 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.817930    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k" event={"ID":"bd87e327-135e-4a72-9925-4d28d4a9360a","Type":"ContainerDied","Data":"f0f749f58621bcc35dc7d55dc5b2fecf52ecf7fd4a924a39f8a4ba53ba63989e"}
Nov 04 01:58:56.818017 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.817950    5039 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0f749f58621bcc35dc7d55dc5b2fecf52ecf7fd4a924a39f8a4ba53ba63989e"
Nov 04 01:58:56.819142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.819123    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:58:56.825847 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.825832    5039 scope.go:122] "RemoveContainer" containerID="4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15"
Nov 04 01:58:56.826101 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:58:56.826084    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15\": container with ID starting with 4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15 not found: ID does not exist" containerID="4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15"
Nov 04 01:58:56.826145 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.826109    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15"} err="failed to get container status \"4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15\": rpc error: code = NotFound desc = could not find container \"4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15\": container with ID starting with 4bf65d8f82f402eff81f2739e88ccf5c4a56b7b520f6733e26bc5e50a7760a15 not found: ID does not exist"
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891590    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891659    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891719    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891760    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891799    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891828    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891906    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891939    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.891968    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892004    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892023    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892042    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892070    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert\") pod \"bd87e327-135e-4a72-9925-4d28d4a9360a\" (UID: \"bd87e327-135e-4a72-9925-4d28d4a9360a\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892114    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892139    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892181    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892217    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892289    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs\") pod \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\" (UID: \"52880f5a-85cd-403f-95e1-c1f08a3bb9bc\") "
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892778    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.893286 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.892969    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.894408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.893824    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.894408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.894063    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:56.894408 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.894187    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.895136 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.895056    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.895242 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.895206    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.895303 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.895242    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.895392 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.895371    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.895586 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.895548    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:56.897932 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.897907    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.898006 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.897954    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "kube-api-access-k4j5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.898814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.898794    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:56.898814 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.898793    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.898958 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.898892    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.899106 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.899086    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "kube-api-access-2ht4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.899218 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.899196    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token" pod "bd87e327-135e-4a72-9925-4d28d4a9360a" (UID: "bd87e327-135e-4a72-9925-4d28d4a9360a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.899465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.899435    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo" pod "52880f5a-85cd-403f-95e1-c1f08a3bb9bc" (UID: "52880f5a-85cd-403f-95e1-c1f08a3bb9bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:56.993496 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993469    5039 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993496 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993493    5039 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993496 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993503    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4j5b\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-kube-api-access-k4j5b\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993511    5039 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993520    5039 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993528    5039 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993536    5039 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993544    5039 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993552    5039 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993559    5039 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993579    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ht4n\" (UniqueName: \"kubernetes.io/projected/bd87e327-135e-4a72-9925-4d28d4a9360a-kube-api-access-2ht4n\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993587    5039 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993595    5039 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd87e327-135e-4a72-9925-4d28d4a9360a-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993604    5039 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd87e327-135e-4a72-9925-4d28d4a9360a-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993612    5039 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993621    5039 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993629    5039 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.993795 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:56.993638    5039 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52880f5a-85cd-403f-95e1-c1f08a3bb9bc-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.247657 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:57.247603    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:57.249991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:57.249969    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"]
Nov 04 01:58:57.253409 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:57.253382    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq"]
Nov 04 01:58:57.406347 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:57.406269    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" path="/var/lib/kubelet/pods/52880f5a-85cd-403f-95e1-c1f08a3bb9bc/volumes"
Nov 04 01:58:57.824530 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:57.824505    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"
Nov 04 01:58:58.005613 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.005561    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"]
Nov 04 01:58:58.012982 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.012953    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k"]
Nov 04 01:58:58.077736 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.077706    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:58.209716 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209688    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.209917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209774    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.209917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209804    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.209917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209843    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.209917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209877    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.209917 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209906    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.210158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209953    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.210158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.209997    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.210158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210011    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.210158 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210109    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs\") pod \"20cca625-41e3-4b6b-937f-2a9044e5ff81\" (UID: \"20cca625-41e3-4b6b-937f-2a9044e5ff81\") "
Nov 04 01:58:58.210334 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210241    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.210756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210410    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.210756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210421    5039 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.210756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210462    5039 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.210756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210645    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.210756 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.210727    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:58.212862 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.212830    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.212967 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.212928    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:58.213337 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.213317    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:58.213410 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.213375    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch" pod "20cca625-41e3-4b6b-937f-2a9044e5ff81" (UID: "20cca625-41e3-4b6b-937f-2a9044e5ff81"). InnerVolumeSpecName "kube-api-access-gwnch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310912    5039 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310939    5039 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cca625-41e3-4b6b-937f-2a9044e5ff81-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310948    5039 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310956    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwnch\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-kube-api-access-gwnch\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310966    5039 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310975    5039 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cca625-41e3-4b6b-937f-2a9044e5ff81-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.310986 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.310983    5039 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cca625-41e3-4b6b-937f-2a9044e5ff81-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.833070 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.833025    5039 generic.go:356] "Generic (PLEG): container finished" podID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerID="7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5" exitCode=0
Nov 04 01:58:58.833510 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.833087    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"
Nov 04 01:58:58.833510 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.833106    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" event={"ID":"20cca625-41e3-4b6b-937f-2a9044e5ff81","Type":"ContainerDied","Data":"7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5"}
Nov 04 01:58:58.833510 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.833173    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p" event={"ID":"20cca625-41e3-4b6b-937f-2a9044e5ff81","Type":"ContainerDied","Data":"4b0ccff1b9b361d7356dc2f3fac24b12495c40275b397483d0338e2ddef9bedd"}
Nov 04 01:58:58.833510 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.833197    5039 scope.go:122] "RemoveContainer" containerID="7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5"
Nov 04 01:58:58.842755 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.842735    5039 scope.go:122] "RemoveContainer" containerID="7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5"
Nov 04 01:58:58.843081 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:58:58.843056    5039 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5\": container with ID starting with 7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5 not found: ID does not exist" containerID="7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5"
Nov 04 01:58:58.843175 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.843092    5039 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5"} err="failed to get container status \"7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5\": rpc error: code = NotFound desc = could not find container \"7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5\": container with ID starting with 7485d40bf0831a8974b5066f952ae9a7926aed7bab90cb683426bbd758c79fe5 not found: ID does not exist"
Nov 04 01:58:58.997043 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:58.997015    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"]
Nov 04 01:58:59.047609 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:59.047557    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p"]
Nov 04 01:58:59.405995 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:59.405962    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" path="/var/lib/kubelet/pods/20cca625-41e3-4b6b-937f-2a9044e5ff81/volumes"
Nov 04 01:58:59.406592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:58:59.406555    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" path="/var/lib/kubelet/pods/bd87e327-135e-4a72-9925-4d28d4a9360a/volumes"
Nov 04 01:59:02.040211 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:02.040176    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:04.401494 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:04.401457    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:06.819086 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:06.819040    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:11.650392 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:11.650316    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:14.400996 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:14.400960    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:16.422639 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:16.422602    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:17.400706 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:17.400667    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:21.235604 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:21.235543    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:22.751156 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:59:22.751125    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:59:25.954873 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:25.954838    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:27.401302 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:27.401269    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:28.086190 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:28.086156    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:29.567212 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:29.567185    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:59:29.573353 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:29.573329    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wsc5z_b546556a-5219-406b-88d0-a3b490c7b259/console-operator/2.log"
Nov 04 01:59:29.768773 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:29.768745    5039 scope.go:122] "RemoveContainer" containerID="3f26adaa8045ece0db8d30f637144146c2330ae042c6390508e221474aa2f109"
Nov 04 01:59:30.745131 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:30.745094    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:31.532328 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:31.532292    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:31.758109 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:31.758074    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"]
Nov 04 01:59:31.760072 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:31.760041    5039 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:31.780014 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:31.779989    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:32.026772 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:32.026736    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:33.801706 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:33.801675    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:35.534740 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:35.534709    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:36.758927 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:36.758888    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing" containerID="cri-o://6d622152d7fdc07a61966624917e8036da7fd48513fc1db5495ab03b1b9a1372" gracePeriod=25
Nov 04 01:59:37.036417 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.036386    5039 generic.go:356] "Generic (PLEG): container finished" podID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerID="6d622152d7fdc07a61966624917e8036da7fd48513fc1db5495ab03b1b9a1372" exitCode=0
Nov 04 01:59:37.036635 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.036426    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" event={"ID":"6a3fb56a-6e1e-4569-9d88-6bcc688e8322","Type":"ContainerDied","Data":"6d622152d7fdc07a61966624917e8036da7fd48513fc1db5495ab03b1b9a1372"}
Nov 04 01:59:37.056952 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.056926    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:59:37.155484 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.155449    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume\") pod \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") "
Nov 04 01:59:37.155779 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.155758    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials\") pod \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") "
Nov 04 01:59:37.155921 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.155904    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume\") pod \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") "
Nov 04 01:59:37.156069 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.156055    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq\") pod \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\" (UID: \"6a3fb56a-6e1e-4569-9d88-6bcc688e8322\") "
Nov 04 01:59:37.157117 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.157063    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume" pod "6a3fb56a-6e1e-4569-9d88-6bcc688e8322" (UID: "6a3fb56a-6e1e-4569-9d88-6bcc688e8322"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:37.157991 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.157970    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume" pod "6a3fb56a-6e1e-4569-9d88-6bcc688e8322" (UID: "6a3fb56a-6e1e-4569-9d88-6bcc688e8322"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:37.159901 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.159878    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq" pod "6a3fb56a-6e1e-4569-9d88-6bcc688e8322" (UID: "6a3fb56a-6e1e-4569-9d88-6bcc688e8322"). InnerVolumeSpecName "kube-api-access-c57jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:37.161088 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.161044    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials" pod "6a3fb56a-6e1e-4569-9d88-6bcc688e8322" (UID: "6a3fb56a-6e1e-4569-9d88-6bcc688e8322"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:37.257880 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.257844    5039 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.258142 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.258123    5039 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-provider-credentials\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.258283 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.258268    5039 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-routing-overlay-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.258412 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:37.258398    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c57jq\" (UniqueName: \"kubernetes.io/projected/6a3fb56a-6e1e-4569-9d88-6bcc688e8322-kube-api-access-c57jq\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:38.055507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:38.054358    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j" event={"ID":"6a3fb56a-6e1e-4569-9d88-6bcc688e8322","Type":"ContainerDied","Data":"35cd0539f5af0e74585bda79d509d74e475493faa4c524ccd25676f8c09dfc5b"}
Nov 04 01:59:38.055507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:38.054431    5039 scope.go:122] "RemoveContainer" containerID="6d622152d7fdc07a61966624917e8036da7fd48513fc1db5495ab03b1b9a1372"
Nov 04 01:59:38.055507 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:38.054610    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"
Nov 04 01:59:38.217465 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:38.217434    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"]
Nov 04 01:59:38.270592 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:38.268420    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j"]
Nov 04 01:59:38.873124 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:38.872870    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:39.406541 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:39.406514    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" path="/var/lib/kubelet/pods/6a3fb56a-6e1e-4569-9d88-6bcc688e8322/volumes"
Nov 04 01:59:40.535298 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:40.535265    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:45.408522 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:45.408485    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:45.659146 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:45.659062    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:50.132057 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:50.132023    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:51.555528 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:51.555493    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:52.430098 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:52.430071    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:52.675794 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:52.675725    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:52.750853 ip-10-0-140-18 kubenswrapper[5039]: E1104 01:59:52.750782    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 01:59:54.824151 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:54.824114    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:55.172561 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:55.172524    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:57.386893 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:57.386862    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:57.671600 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:57.671509    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:58.005914 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:58.005812    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:58.949588 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:58.949535    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 01:59:59.614463 ip-10-0-140-18 kubenswrapper[5039]: I1104 01:59:59.614423    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:00.155034 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.154992    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"]
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155405    5039 cpu_manager.go:414] "containerMap: removing container" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155416    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155426    5039 cpu_manager.go:414] "containerMap: removing container" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155431    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155460    5039 cpu_manager.go:414] "containerMap: removing container" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155465    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155480    5039 cpu_manager.go:414] "containerMap: removing container" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy"
Nov 04 02:00:00.155529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155484    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy"
Nov 04 02:00:00.156000 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155537    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="52880f5a-85cd-403f-95e1-c1f08a3bb9bc" containerName="istio-proxy"
Nov 04 02:00:00.156000 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155546    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="20cca625-41e3-4b6b-937f-2a9044e5ff81" containerName="istio-proxy"
Nov 04 02:00:00.156000 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155554    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a3fb56a-6e1e-4569-9d88-6bcc688e8322" containerName="payload-processing"
Nov 04 02:00:00.156000 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.155559    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd87e327-135e-4a72-9925-4d28d4a9360a" containerName="istio-proxy"
Nov 04 02:00:00.158784 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.158765    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.211015 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.210987    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 02:00:00.262259 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.262224    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"]
Nov 04 02:00:00.315883 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.315840    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5lj\" (UniqueName: \"kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj\") pod \"maas-api-key-cleanup-29895960-pxjpp\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.316075 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.315901    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895960-pxjpp\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.417300 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.417207    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895960-pxjpp\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.417476 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.417323    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5lj\" (UniqueName: \"kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj\") pod \"maas-api-key-cleanup-29895960-pxjpp\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.417970 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.417945    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895960-pxjpp\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.477890 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.477863    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5lj\" (UniqueName: \"kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj\") pod \"maas-api-key-cleanup-29895960-pxjpp\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:00.773330 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:00.773254    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:01.153314 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:01.153285    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"]
Nov 04 02:00:01.155197 ip-10-0-140-18 kubenswrapper[5039]: W1104 02:00:01.155172    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96917ad_9cae_4f98_9cf2_023ebd728f33.slice/crio-d667db4f15bd8bae1e1a1f01a9a2e8b471a5650140f638794302bd97161d79b5 WatchSource:0}: Error finding container d667db4f15bd8bae1e1a1f01a9a2e8b471a5650140f638794302bd97161d79b5: Status 404 returned error can't find the container with id d667db4f15bd8bae1e1a1f01a9a2e8b471a5650140f638794302bd97161d79b5
Nov 04 02:00:01.157451 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:01.157433    5039 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:00:01.188938 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:01.188899    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" event={"ID":"c96917ad-9cae-4f98-9cf2-023ebd728f33","Type":"ContainerStarted","Data":"d667db4f15bd8bae1e1a1f01a9a2e8b471a5650140f638794302bd97161d79b5"}
Nov 04 02:00:02.142535 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.142504    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"]
Nov 04 02:00:02.147379 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.147358    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.200442 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.200414    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-7af38e-openshift-default-dockercfg-pbl8c\""
Nov 04 02:00:02.236551 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236519    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.236551 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236556    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.236805 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236611    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.236805 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236718    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.236805 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236751    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.236805 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236773    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.237005 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236889    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.237005 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.236940    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9wjf\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.237111 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.237032    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.251768 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.251744    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"]
Nov 04 02:00:02.338424 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338385    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338636 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338439    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338636 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338468    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338636 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338527    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338636 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338589    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9wjf\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338848 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338662    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338848 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338690    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338848 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338716    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.338848 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338768    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.339048 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338844    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.339048 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338860    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.339048 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.338929    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.339255 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.339228    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.339450 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.339432    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.341514 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.341484    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.341889 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.341866    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.399071 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.399005    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.399071 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.399060    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9wjf\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.460151 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.460121    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:02.886204 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.886178    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"]
Nov 04 02:00:02.886725 ip-10-0-140-18 kubenswrapper[5039]: W1104 02:00:02.886701    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d2bdca_2a29_47c4_8e3f_23a5423c6ba9.slice/crio-ee419e95de074a0506bc794302895592f5079f5b2e47a8c012874d905d6683e4 WatchSource:0}: Error finding container ee419e95de074a0506bc794302895592f5079f5b2e47a8c012874d905d6683e4: Status 404 returned error can't find the container with id ee419e95de074a0506bc794302895592f5079f5b2e47a8c012874d905d6683e4
Nov 04 02:00:02.889040 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.889006    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 02:00:02.889128 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.889079    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 02:00:02.889128 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:02.889118    5039 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 02:00:03.206079 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:03.205996    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" event={"ID":"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9","Type":"ContainerStarted","Data":"dbc4334c6a4179ceca9e5b7703103a8dda9dcde3b4867f79efe1b30672e3b50a"}
Nov 04 02:00:03.206079 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:03.206051    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" event={"ID":"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9","Type":"ContainerStarted","Data":"ee419e95de074a0506bc794302895592f5079f5b2e47a8c012874d905d6683e4"}
Nov 04 02:00:03.461248 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:03.461165    5039 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:03.462835 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:03.462805    5039 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 02:00:03.462966 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:03.462867    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 02:00:04.018850 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:04.018816    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:04.271857 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:04.271777    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:04.461102 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:04.461070    5039 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 02:00:04.461266 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:04.461122    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 02:00:04.518474 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:04.518444    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:05.460754 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:05.460719    5039 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 02:00:05.461150 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:05.460780    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 02:00:05.943660 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:05.941085    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:06.192096 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:06.192059    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:06.461244 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:06.461212    5039 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 02:00:06.461640 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:06.461267    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 02:00:07.461256 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:07.461220    5039 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Nov 04 02:00:07.461793 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:07.461279    5039 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Nov 04 02:00:07.506358 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:07.506327    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:07.949599 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:07.946471    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podStartSLOduration=5.946452453 podStartE2EDuration="5.946452453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:03.328323471 +0000 UTC m=+1534.044510327" watchObservedRunningTime="2026-11-04 02:00:07.946452453 +0000 UTC m=+1538.662639309"
Nov 04 02:00:07.949599 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:07.946999    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"]
Nov 04 02:00:07.949599 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:07.947203    5039 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy" containerID="cri-o://dbc4334c6a4179ceca9e5b7703103a8dda9dcde3b4867f79efe1b30672e3b50a" gracePeriod=30
Nov 04 02:00:09.030297 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:09.030269    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65457c7d56-kdp2h_ae5536f6-a4af-4705-b5ea-8177e0bfead1/authorino/0.log"
Nov 04 02:00:09.077838 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:09.077749    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:09.424020 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:09.423959    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-9x6jf_4a6dbbf5-313a-4ed8-afb6-6e1a94f86033/manager/0.log"
Nov 04 02:00:09.667624 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:09.667597    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-kj776_5049d2db-2b1f-4df4-b779-db8213773d63/manager/0.log"
Nov 04 02:00:10.276804 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:10.276768    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" event={"ID":"c96917ad-9cae-4f98-9cf2-023ebd728f33","Type":"ContainerStarted","Data":"a82d1c8b6503f2c60e18cb8a76ca457feaa335da8df1bf51a824d6ddc79d01e9"}
Nov 04 02:00:10.392756 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:10.392710    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" podStartSLOduration=1.9985442039999999 podStartE2EDuration="10.392696005s" totalImagesPullingTime="8.394151801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:10.390421031 +0000 UTC m=+1541.106607888" watchObservedRunningTime="2026-11-04 02:00:10.392696005 +0000 UTC m=+1541.108882863"
Nov 04 02:00:10.899980 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:10.899945    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cjjc7_14205fbd-03a1-442b-9907-4364028e4a24/manager/0.log"
Nov 04 02:00:11.283676 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:11.283562    5039 generic.go:356] "Generic (PLEG): container finished" podID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerID="a82d1c8b6503f2c60e18cb8a76ca457feaa335da8df1bf51a824d6ddc79d01e9" exitCode=60
Nov 04 02:00:11.283676 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:11.283655    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" event={"ID":"c96917ad-9cae-4f98-9cf2-023ebd728f33","Type":"ContainerDied","Data":"a82d1c8b6503f2c60e18cb8a76ca457feaa335da8df1bf51a824d6ddc79d01e9"}
Nov 04 02:00:11.284138 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:11.283928    5039 scope.go:122] "RemoveContainer" containerID="a82d1c8b6503f2c60e18cb8a76ca457feaa335da8df1bf51a824d6ddc79d01e9"
Nov 04 02:00:12.290334 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:12.290252    5039 generic.go:356] "Generic (PLEG): container finished" podID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerID="c0f05963e2c3392878b795b4216179723273849709864a677c1ce5957ee3d4c4" exitCode=60
Nov 04 02:00:12.290334 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:12.290314    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" event={"ID":"c96917ad-9cae-4f98-9cf2-023ebd728f33","Type":"ContainerDied","Data":"c0f05963e2c3392878b795b4216179723273849709864a677c1ce5957ee3d4c4"}
Nov 04 02:00:12.290840 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:12.290367    5039 scope.go:122] "RemoveContainer" containerID="a82d1c8b6503f2c60e18cb8a76ca457feaa335da8df1bf51a824d6ddc79d01e9"
Nov 04 02:00:12.290840 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:12.290558    5039 scope.go:122] "RemoveContainer" containerID="c0f05963e2c3392878b795b4216179723273849709864a677c1ce5957ee3d4c4"
Nov 04 02:00:12.290840 ip-10-0-140-18 kubenswrapper[5039]: E1104 02:00:12.290812    5039 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895960-pxjpp_odh-ai-gateway-infra(c96917ad-9cae-4f98-9cf2-023ebd728f33)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33"
Nov 04 02:00:12.716625 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:12.716591    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:13.195176 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:13.195149    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gzdl5_8186cb9e-da9d-4f07-a3cd-a903e1cc7dce/manager/0.log"
Nov 04 02:00:13.297234 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:13.297211    5039 scope.go:122] "RemoveContainer" containerID="c0f05963e2c3392878b795b4216179723273849709864a677c1ce5957ee3d4c4"
Nov 04 02:00:13.297612 ip-10-0-140-18 kubenswrapper[5039]: E1104 02:00:13.297406    5039 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895960-pxjpp_odh-ai-gateway-infra(c96917ad-9cae-4f98-9cf2-023ebd728f33)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33"
Nov 04 02:00:13.462072 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:13.462003    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-wbtv2_e20f2002-f433-45ac-8e69-8de60bde3443/manager/0.log"
Nov 04 02:00:14.206250 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:14.206223    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gstds_5d3c9ac2-e931-4dd6-8871-e4bfdf719c05/manager/0.log"
Nov 04 02:00:15.054726 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.054699    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/1.log"
Nov 04 02:00:15.310831 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.310762    5039 generic.go:356] "Generic (PLEG): container finished" podID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerID="dbc4334c6a4179ceca9e5b7703103a8dda9dcde3b4867f79efe1b30672e3b50a" exitCode=0
Nov 04 02:00:15.310956 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.310834    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" event={"ID":"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9","Type":"ContainerDied","Data":"dbc4334c6a4179ceca9e5b7703103a8dda9dcde3b4867f79efe1b30672e3b50a"}
Nov 04 02:00:15.419869 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.419846    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:15.453857 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.453832    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.453979 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.453865    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.453979 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.453896    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.453979 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.453924    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.454142 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454002    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.454142 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454052    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.454142 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454082    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.454142 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454124    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.454346 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454115    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:15.454346 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454154    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy\") pod \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\" (UID: \"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9\") "
Nov 04 02:00:15.454346 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454185    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:15.454346 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454265    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:15.454549 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454401    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:00:15.454549 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454443    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:15.454549 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454494    5039 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.454549 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454511    5039 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.454549 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454520    5039 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.454549 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.454528    5039 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.456338 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.456315    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:00:15.456451 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.456407    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:15.456691 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.456667    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "kube-api-access-v9wjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:15.456751 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.456703    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token" pod "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" (UID: "d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:15.555072 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.555041    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9wjf\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-kube-api-access-v9wjf\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.555183 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.555074    5039 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.555183 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.555092    5039 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.555183 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.555106    5039 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:15.555183 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:15.555119    5039 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:16.317832 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:16.317794    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj" event={"ID":"d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9","Type":"ContainerDied","Data":"ee419e95de074a0506bc794302895592f5079f5b2e47a8c012874d905d6683e4"}
Nov 04 02:00:16.317832 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:16.317830    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"
Nov 04 02:00:16.318317 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:16.317865    5039 scope.go:122] "RemoveContainer" containerID="dbc4334c6a4179ceca9e5b7703103a8dda9dcde3b4867f79efe1b30672e3b50a"
Nov 04 02:00:16.530757 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:16.530734    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"]
Nov 04 02:00:16.580154 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:16.580088    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj"]
Nov 04 02:00:17.405497 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:17.405466    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" path="/var/lib/kubelet/pods/d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9/volumes"
Nov 04 02:00:17.648085 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:17.648057    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/0.log"
Nov 04 02:00:22.400241 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:22.400208    5039 scope.go:122] "RemoveContainer" containerID="c0f05963e2c3392878b795b4216179723273849709864a677c1ce5957ee3d4c4"
Nov 04 02:00:22.461529 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:22.461503    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65457c7d56-kdp2h_ae5536f6-a4af-4705-b5ea-8177e0bfead1/authorino/0.log"
Nov 04 02:00:22.751069 ip-10-0-140-18 kubenswrapper[5039]: E1104 02:00:22.751040    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 02:00:23.354689 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:23.354656    5039 generic.go:356] "Generic (PLEG): container finished" podID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerID="540266fe4345fd11c4bfa9041a4092f293c4c466fdb97b054839ebdace03fb51" exitCode=60
Nov 04 02:00:23.354781 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:23.354728    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" event={"ID":"c96917ad-9cae-4f98-9cf2-023ebd728f33","Type":"ContainerDied","Data":"540266fe4345fd11c4bfa9041a4092f293c4c466fdb97b054839ebdace03fb51"}
Nov 04 02:00:23.354829 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:23.354789    5039 scope.go:122] "RemoveContainer" containerID="c0f05963e2c3392878b795b4216179723273849709864a677c1ce5957ee3d4c4"
Nov 04 02:00:23.355105 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:23.355092    5039 scope.go:122] "RemoveContainer" containerID="540266fe4345fd11c4bfa9041a4092f293c4c466fdb97b054839ebdace03fb51"
Nov 04 02:00:23.355332 ip-10-0-140-18 kubenswrapper[5039]: E1104 02:00:23.355313    5039 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895960-pxjpp_odh-ai-gateway-infra(c96917ad-9cae-4f98-9cf2-023ebd728f33)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33"
Nov 04 02:00:24.479651 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.479618    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"]
Nov 04 02:00:24.614397 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.614374    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:24.630160 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.630138    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj\" (UniqueName: \"kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj\") pod \"c96917ad-9cae-4f98-9cf2-023ebd728f33\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") "
Nov 04 02:00:24.630266 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.630181    5039 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca\") pod \"c96917ad-9cae-4f98-9cf2-023ebd728f33\" (UID: \"c96917ad-9cae-4f98-9cf2-023ebd728f33\") "
Nov 04 02:00:24.630552 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.630534    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca" pod "c96917ad-9cae-4f98-9cf2-023ebd728f33" (UID: "c96917ad-9cae-4f98-9cf2-023ebd728f33"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:00:24.632191 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.632173    5039 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj" pod "c96917ad-9cae-4f98-9cf2-023ebd728f33" (UID: "c96917ad-9cae-4f98-9cf2-023ebd728f33"). InnerVolumeSpecName "kube-api-access-lq5lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:24.731762 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.731695    5039 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq5lj\" (UniqueName: \"kubernetes.io/projected/c96917ad-9cae-4f98-9cf2-023ebd728f33-kube-api-access-lq5lj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:24.731762 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:24.731727    5039 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96917ad-9cae-4f98-9cf2-023ebd728f33-maas-api-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:25.369551 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:25.369516    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp" event={"ID":"c96917ad-9cae-4f98-9cf2-023ebd728f33","Type":"ContainerDied","Data":"d667db4f15bd8bae1e1a1f01a9a2e8b471a5650140f638794302bd97161d79b5"}
Nov 04 02:00:25.369551 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:25.369535    5039 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"
Nov 04 02:00:25.369805 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:25.369590    5039 scope.go:122] "RemoveContainer" containerID="540266fe4345fd11c4bfa9041a4092f293c4c466fdb97b054839ebdace03fb51"
Nov 04 02:00:25.532578 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:25.532535    5039 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"]
Nov 04 02:00:25.537004 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:25.536983    5039 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895960-pxjpp"]
Nov 04 02:00:27.404847 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:27.404814    5039 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" path="/var/lib/kubelet/pods/c96917ad-9cae-4f98-9cf2-023ebd728f33/volumes"
Nov 04 02:00:28.538075 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:28.538037    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:39.174208 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:39.174173    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gzdl5_8186cb9e-da9d-4f07-a3cd-a903e1cc7dce/manager/0.log"
Nov 04 02:00:39.426984 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:39.426911    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-wbtv2_e20f2002-f433-45ac-8e69-8de60bde3443/manager/0.log"
Nov 04 02:00:39.719623 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:39.719505    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gstds_5d3c9ac2-e931-4dd6-8871-e4bfdf719c05/manager/0.log"
Nov 04 02:00:40.049705 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:40.049580    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/1.log"
Nov 04 02:00:41.589475 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:41.589443    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/0.log"
Nov 04 02:00:44.620302 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:44.620238    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-b8gg8_a08c10ca-6078-4eee-8382-63a36f82533d/postgres/0.log"
Nov 04 02:00:46.695474 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:46.695442    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65457c7d56-kdp2h_ae5536f6-a4af-4705-b5ea-8177e0bfead1/authorino/0.log"
Nov 04 02:00:47.079496 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:47.076478    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-9x6jf_4a6dbbf5-313a-4ed8-afb6-6e1a94f86033/manager/0.log"
Nov 04 02:00:47.322873 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:47.322843    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-kj776_5049d2db-2b1f-4df4-b779-db8213773d63/manager/0.log"
Nov 04 02:00:48.349161 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:48.349124    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cjjc7_14205fbd-03a1-442b-9907-4364028e4a24/manager/0.log"
Nov 04 02:00:51.197769 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:51.197715    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-kdtlh_e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f/istio-proxy/0.log"
Nov 04 02:00:51.428841 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:51.428811    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:52.738167 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:52.738141    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-qpptj_dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc/istio-proxy/0.log"
Nov 04 02:00:52.751480 ip-10-0-140-18 kubenswrapper[5039]: E1104 02:00:52.751450    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 02:00:53.210444 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:53.210413    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-vkm88_dbeaf941-77d4-45f8-8e94-825323dc4a96/istio-proxy/1.log"
Nov 04 02:00:53.934410 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:53.934377    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:00:54.174763 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:54.174734    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54848578cf-gq6dc_07ebf665-177c-48e7-969c-28057a4ff1a4/router/0.log"
Nov 04 02:00:54.912364 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:00:54.912335    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-vkm88_dbeaf941-77d4-45f8-8e94-825323dc4a96/istio-proxy/0.log"
Nov 04 02:01:08.873428 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873397    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t4ldv/must-gather-sfptm"]
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873805    5039 cpu_manager.go:414] "containerMap: removing container" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873821    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873850    5039 cpu_manager.go:414] "containerMap: removing container" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873855    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873869    5039 cpu_manager.go:414] "containerMap: removing container" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873875    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873955    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873964    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.873972    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9d2bdca-2a29-47c4-8e3f-23a5423c6ba9" containerName="istio-proxy"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.874055    5039 cpu_manager.go:414] "containerMap: removing container" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.874060    5039 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.875916 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.874114    5039 memory_manager.go:365] "RemoveStaleState removing state" podUID="c96917ad-9cae-4f98-9cf2-023ebd728f33" containerName="cleanup"
Nov 04 02:01:08.876896 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.876879    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:08.886761 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.886739    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:08.938557 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.938527    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4ldv/must-gather-sfptm"]
Nov 04 02:01:08.984788 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.984765    5039 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t4ldv\"/\"default-dockercfg-4lxxc\""
Nov 04 02:01:08.984788 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.984771    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4ldv\"/\"kube-root-ca.crt\""
Nov 04 02:01:08.984788 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.984783    5039 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4ldv\"/\"openshift-service-ca.crt\""
Nov 04 02:01:08.991306 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.991284    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx2zx\" (UniqueName: \"kubernetes.io/projected/a9533d51-d41b-4bd8-a7db-f3f55e9d38af-kube-api-access-gx2zx\") pod \"must-gather-sfptm\" (UID: \"a9533d51-d41b-4bd8-a7db-f3f55e9d38af\") " pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:08.991456 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:08.991425    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9533d51-d41b-4bd8-a7db-f3f55e9d38af-must-gather-output\") pod \"must-gather-sfptm\" (UID: \"a9533d51-d41b-4bd8-a7db-f3f55e9d38af\") " pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:09.092641 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.092617    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx2zx\" (UniqueName: \"kubernetes.io/projected/a9533d51-d41b-4bd8-a7db-f3f55e9d38af-kube-api-access-gx2zx\") pod \"must-gather-sfptm\" (UID: \"a9533d51-d41b-4bd8-a7db-f3f55e9d38af\") " pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:09.092785 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.092699    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9533d51-d41b-4bd8-a7db-f3f55e9d38af-must-gather-output\") pod \"must-gather-sfptm\" (UID: \"a9533d51-d41b-4bd8-a7db-f3f55e9d38af\") " pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:09.092986 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.092972    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a9533d51-d41b-4bd8-a7db-f3f55e9d38af-must-gather-output\") pod \"must-gather-sfptm\" (UID: \"a9533d51-d41b-4bd8-a7db-f3f55e9d38af\") " pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:09.189013 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.188952    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx2zx\" (UniqueName: \"kubernetes.io/projected/a9533d51-d41b-4bd8-a7db-f3f55e9d38af-kube-api-access-gx2zx\") pod \"must-gather-sfptm\" (UID: \"a9533d51-d41b-4bd8-a7db-f3f55e9d38af\") " pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:09.190862 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.190845    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4ldv/must-gather-sfptm"
Nov 04 02:01:09.226853 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.226820    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:09.570617 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.570596    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4ldv/must-gather-sfptm"]
Nov 04 02:01:09.572714 ip-10-0-140-18 kubenswrapper[5039]: W1104 02:01:09.572688    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9533d51_d41b_4bd8_a7db_f3f55e9d38af.slice/crio-e6f86dba955906f949c1cc90c274047e35b257c97e1a0277d1a750eb9f1ee1a3 WatchSource:0}: Error finding container e6f86dba955906f949c1cc90c274047e35b257c97e1a0277d1a750eb9f1ee1a3: Status 404 returned error can't find the container with id e6f86dba955906f949c1cc90c274047e35b257c97e1a0277d1a750eb9f1ee1a3
Nov 04 02:01:09.579798 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:09.579773    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/must-gather-sfptm" event={"ID":"a9533d51-d41b-4bd8-a7db-f3f55e9d38af","Type":"ContainerStarted","Data":"e6f86dba955906f949c1cc90c274047e35b257c97e1a0277d1a750eb9f1ee1a3"}
Nov 04 02:01:11.598342 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:11.598301    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/must-gather-sfptm" event={"ID":"a9533d51-d41b-4bd8-a7db-f3f55e9d38af","Type":"ContainerStarted","Data":"9b3b2d2d939754a358af9dff6c8ac865ce97d285fc4e826e1b8769520f743002"}
Nov 04 02:01:11.598744 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:11.598354    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/must-gather-sfptm" event={"ID":"a9533d51-d41b-4bd8-a7db-f3f55e9d38af","Type":"ContainerStarted","Data":"5fb5eae3b41a3cf8fd4b1c2fdb2394adcfb5d27cb4f8533ee1fdba0868b5e722"}
Nov 04 02:01:11.716729 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:11.716672    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-t4ldv/must-gather-sfptm" podStartSLOduration=2.4186456769999998 podStartE2EDuration="3.716658087s" totalImagesPullingTime="1.29801241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:11.711407424 +0000 UTC m=+1602.427594280" watchObservedRunningTime="2026-11-04 02:01:11.716658087 +0000 UTC m=+1602.432845017"
Nov 04 02:01:18.333904 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:18.333871    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p7jkf_76b11319-689c-4ba5-83ec-afe254b80174/global-pull-secret-syncer/0.log"
Nov 04 02:01:19.578429 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:19.574929    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-f66h9_6a1464fb-a344-4bd9-8406-6fe464ff4fa0/connection-checker/0.log"
Nov 04 02:01:20.007759 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:20.007733    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-j4l4l_81cc5d87-5a2f-43e5-ba84-04993c15476e/connection-checker/0.log"
Nov 04 02:01:20.477176 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:20.477135    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-sx466_fcc699c1-3d13-46f0-8f81-fcb9f7c7a1db/connection-checker/0.log"
Nov 04 02:01:20.975090 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:20.975058    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log"
Nov 04 02:01:21.007926 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:21.007900    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gzdl5_8186cb9e-da9d-4f07-a3cd-a903e1cc7dce/manager/0.log"
Nov 04 02:01:21.247230 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:21.247146    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-wbtv2_e20f2002-f433-45ac-8e69-8de60bde3443/manager/0.log"
Nov 04 02:01:21.558895 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:21.558701    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gstds_5d3c9ac2-e931-4dd6-8871-e4bfdf719c05/manager/0.log"
Nov 04 02:01:21.725138 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:21.725107    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/0.log"
Nov 04 02:01:21.809636 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:21.807751    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/1.log"
Nov 04 02:01:22.753172 ip-10-0-140-18 kubenswrapper[5039]: E1104 02:01:22.753134    5039 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6xjwj" containerName="extractor"
Nov 04 02:01:23.221590 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:23.218814    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal_aae6ff4840f4ffd5842f280d0afa74f6/haproxy/0.log"
Nov 04 02:01:25.817597 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:25.817424    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gzdl5_8186cb9e-da9d-4f07-a3cd-a903e1cc7dce/manager/0.log"
Nov 04 02:01:26.032622 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:26.032588    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-wbtv2_e20f2002-f433-45ac-8e69-8de60bde3443/manager/0.log"
Nov 04 02:01:26.390951 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:26.390903    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gstds_5d3c9ac2-e931-4dd6-8871-e4bfdf719c05/manager/0.log"
Nov 04 02:01:26.515114 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:26.515046    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/0.log"
Nov 04 02:01:26.612701 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:26.612669    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/1.log"
Nov 04 02:01:26.895783 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:26.895730    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65457c7d56-kdp2h_ae5536f6-a4af-4705-b5ea-8177e0bfead1/authorino/0.log"
Nov 04 02:01:27.222940 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.222869    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-9x6jf_4a6dbbf5-313a-4ed8-afb6-6e1a94f86033/manager/0.log"
Nov 04 02:01:27.641501 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.641454    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-kj776_5049d2db-2b1f-4df4-b779-db8213773d63/manager/0.log"
Nov 04 02:01:27.823765 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.823729    5039 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"]
Nov 04 02:01:27.831992 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.831963    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.855318 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.855292    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-sys\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.855471 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.855325    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-podres\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.855471 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.855369    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-lib-modules\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.855610 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.855504    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8k7t\" (UniqueName: \"kubernetes.io/projected/80313353-efe1-4f53-985a-bbc02a6bb1f8-kube-api-access-t8k7t\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.855610 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.855545    5039 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-proc\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.915829 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.915743    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"]
Nov 04 02:01:27.956681 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956657    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-lib-modules\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.956850 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956705    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8k7t\" (UniqueName: \"kubernetes.io/projected/80313353-efe1-4f53-985a-bbc02a6bb1f8-kube-api-access-t8k7t\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.956850 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956790    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-lib-modules\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.956850 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956819    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-proc\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.957026 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956857    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-proc\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.957026 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956904    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-sys\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.957026 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.956962    5039 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-podres\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.957026 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.957012    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-sys\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:27.957189 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:27.957092    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80313353-efe1-4f53-985a-bbc02a6bb1f8-podres\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:28.017134 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.017106    5039 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8k7t\" (UniqueName: \"kubernetes.io/projected/80313353-efe1-4f53-985a-bbc02a6bb1f8-kube-api-access-t8k7t\") pod \"perf-node-gather-daemonset-mfc8n\" (UID: \"80313353-efe1-4f53-985a-bbc02a6bb1f8\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:28.147687 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.147660    5039 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:28.528260 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.528237    5039 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"]
Nov 04 02:01:28.530718 ip-10-0-140-18 kubenswrapper[5039]: W1104 02:01:28.530690    5039 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod80313353_efe1_4f53_985a_bbc02a6bb1f8.slice/crio-b7f090444dd8d92f5dcb6904cb8260c68983d5268afe691dc0dd8e05b60c5afe WatchSource:0}: Error finding container b7f090444dd8d92f5dcb6904cb8260c68983d5268afe691dc0dd8e05b60c5afe: Status 404 returned error can't find the container with id b7f090444dd8d92f5dcb6904cb8260c68983d5268afe691dc0dd8e05b60c5afe
Nov 04 02:01:28.747688 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.747649    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n" event={"ID":"80313353-efe1-4f53-985a-bbc02a6bb1f8","Type":"ContainerStarted","Data":"ee42ec154600a112c3d198396ded465abf41203eb344461b3d5ccb5bb2ce63fa"}
Nov 04 02:01:28.747879 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.747747    5039 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n" event={"ID":"80313353-efe1-4f53-985a-bbc02a6bb1f8","Type":"ContainerStarted","Data":"b7f090444dd8d92f5dcb6904cb8260c68983d5268afe691dc0dd8e05b60c5afe"}
Nov 04 02:01:28.747879 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.747839    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:28.779460 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.779392    5039 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-56trq_016762ac-ad73-46b0-8059-9b4b2b670043/konnectivity-agent/0.log" limit=1024
Nov 04 02:01:28.864368 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.864309    5039 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n" podStartSLOduration=1.864293626 podStartE2EDuration="1.864293626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:28.864228829 +0000 UTC m=+1619.580415686" watchObservedRunningTime="2026-11-04 02:01:28.864293626 +0000 UTC m=+1619.580480483"
Nov 04 02:01:28.941537 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:28.941508    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cjjc7_14205fbd-03a1-442b-9907-4364028e4a24/manager/0.log"
Nov 04 02:01:30.630878 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:30.630850    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-gzdl5_8186cb9e-da9d-4f07-a3cd-a903e1cc7dce/manager/0.log"
Nov 04 02:01:30.858118 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:30.858085    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-wbtv2_e20f2002-f433-45ac-8e69-8de60bde3443/manager/0.log"
Nov 04 02:01:31.170474 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:31.170445    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gstds_5d3c9ac2-e931-4dd6-8871-e4bfdf719c05/manager/0.log"
Nov 04 02:01:31.343424 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:31.343386    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/0.log"
Nov 04 02:01:31.395943 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:31.395908    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8677bb4cb4-5d7s9_2bbe71a9-9f3e-4700-8fe2-7462e2ed25e4/manager/1.log"
Nov 04 02:01:34.763530 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:34.763501    5039 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-mfc8n"
Nov 04 02:01:35.195348 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:35.195308    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-j4lwn_8ec90134-2d9f-418f-98e5-877305e1a0b2/cluster-monitoring-operator/0.log"
Nov 04 02:01:37.593414 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:37.593381    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-kdtlh_e6a9a5ce-74d6-479e-ba63-6c79f0e75a1f/istio-proxy/0.log"
Nov 04 02:01:38.885790 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:38.885761    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-qpptj_dbb8bdd3-cd7f-40d0-8886-48ea8c759cdc/istio-proxy/0.log"
Nov 04 02:01:39.319672 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:39.319596    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-vkm88_dbeaf941-77d4-45f8-8e94-825323dc4a96/istio-proxy/0.log"
Nov 04 02:01:39.320223 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:39.320210    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-vkm88_dbeaf941-77d4-45f8-8e94-825323dc4a96/istio-proxy/1.log"
Nov 04 02:01:40.097512 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:40.097488    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54848578cf-gq6dc_07ebf665-177c-48e7-969c-28057a4ff1a4/router/0.log"
Nov 04 02:01:40.239406 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:40.239360    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cvcjq_2256d437-8a82-4a72-acc4-3fc5cb85c9a2/node-exporter/0.log"
Nov 04 02:01:40.663471 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:40.663445    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cvcjq_2256d437-8a82-4a72-acc4-3fc5cb85c9a2/kube-rbac-proxy/0.log"
Nov 04 02:01:41.081017 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:41.080946    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cvcjq_2256d437-8a82-4a72-acc4-3fc5cb85c9a2/init-textfile/0.log"
Nov 04 02:01:43.895423 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:43.895371    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65457c7d56-kdp2h_ae5536f6-a4af-4705-b5ea-8177e0bfead1/authorino/0.log"
Nov 04 02:01:44.010252 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:44.010218    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-9x6jf_4a6dbbf5-313a-4ed8-afb6-6e1a94f86033/manager/0.log"
Nov 04 02:01:44.226828 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:44.226734    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-kj776_5049d2db-2b1f-4df4-b779-db8213773d63/manager/0.log"
Nov 04 02:01:45.025839 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:45.025805    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cjjc7_14205fbd-03a1-442b-9907-4364028e4a24/manager/0.log"
Nov 04 02:01:46.496609 ip-10-0-140-18 kubenswrapper[5039]: I1104 02:01:46.496581    5039 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-cz8v4_2fbdf966-af9e-4705-98bc-94af9a17c460/prometheus-operator-admission-webhook/0.log"